Scribd
Upload
18 views

Data Engineering Course Topics

Uploaded by

venki_hinfotech
Copyright
© © All Rights Reserved
Available Formats
Download as XLSX, PDF, TXT or read online on Scribd
18 views

Data Engineering Course Topics

Uploaded by

venki_hinfotech
Copyright
© © All Rights Reserved
Available Formats
Download as XLSX, PDF, TXT or read online on Scribd
You are on page 1/ 4

Snowflake Architecture.

Snowflake User Interface Walkthrough


Snowflake Administration overview.
Connecting to snowflake (snowsql, Python)
Create Storage integration, External stages, File Formats.
Bulk loading data into snowflake. (COPY INTO Command)
o Loading data from Local machine (user stage,table stage and named stage)
o Loading data from External Stage (AWS S3)
Continuous Data loading using snowpipe.
External tables.
Unloading Data from snowflake.
Data pipeline in snowflake.(real time usecase)
o Snowflake streams
o Snowflake Tasks
Stored procedures (real time usecase)
Time Travel
Cloning Data in snowflake.
Data sharing in snowflake. (realtime usecase)
Snowflake resource monitor.
Working with Metadata Tables.
Materialized Views and Transactions.
Micro partitions and clustering keys.
User defined functions in snowflake
Project Architecture
API integration and external functions
Alternative snowpipe setup
Data Masking
Realtime use cases with python
Introduction to Matilion(Architecture)
Types of Jobs(Orchestration and Transformation Jobs)
Job Variables and Environment Variables
Load/unload components
RDS Query component
SQL Query component
S3 Load component
S3 Unload component
scripting components(Bash and Python)
AWS Components(SNS and SQS)
DDL Components
Create External table component
Create Table component
Refresh External Table component
SQL script component
Flow components(AND,OR,END Success,END Failure)
Iterator components(File Iterator,Table Iterator)
Trasformation Job components(Join,intesect,Table input,Table output,Table Update)
Simple Storage Service
Identity Acccess Management
EC2
SNS
SQS
Lambda functions
RDS
DMS
Python Data structures
Writing functions in Python
snowflake.connector module
boto3 module
psycopg2 module

You might also like