Scribd
Upload
75 views2 pages

Snowflake Hands On

This document provides instructions for loading data from an S3 bucket into Snowflake. It includes: 1. Creating the necessary database, schemas, tables, file formats, and roles in Snowflake using SQL scripts. 2. Loading daily data files from the S3 bucket into the landing layer in Snowflake. The files have names formatted as <filename>_ddmmyyyy.csv and are replaced every 15 minutes. 3. The data will then be moved from the landing layer to the EDW layer for further processing.

Uploaded by

PAVANN T
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
75 views2 pages

Snowflake Hands On

This document provides instructions for loading data from an S3 bucket into Snowflake. It includes: 1. Creating the necessary database, schemas, tables, file formats, and roles in Snowflake using SQL scripts. 2. Loading daily data files from the S3 bucket into the landing layer in Snowflake. The files have names formatted as <filename>_ddmmyyyy.csv and are replaced every 15 minutes. 3. The data will then be moved from the landing layer to the EDW layer for further processing.

Uploaded by

PAVANN T
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

Assumptions:

Dimension Model is ready.

Design is ready

Mapping document is ready

Data is ready and agreed to place in the S3 bucket.

Data provider provides data in files (csv, comma delimited, double quotes encloser)

Retention time in S3 bucket is 15 mins. After 15 mins data from S3 is moved to archive folder

All loads are daily loads

Daily files are uploaded in a S3 bucket

Snowflake Hands on Activities:

1. Create objects:

create all the objects in a .sql file and run it by snowsql

Databases

a) LND_LG b) EDW_LG

Schemas

In LND_LG, schema is SHM_LND_LG

In EDW_LG, schema is SHM_ EDW_LG

Tables

In SHM_LND_LG, create all required tables with LND_ prefix

In SHM_ EDW_LG, create all required tables with EDW_ prefix

Virtual warehouse

Based on the data size

File formats

Supports csv, comma delimited, double quotes encloser

RBAC

Use Account Admin, User Admin roles for RBAC (or) add a custom roles that holds the Account
Admin, User Admin privileges

2. Landing Layer Load

Assumptions: Data is provided in S3 bucket as files. All files are daily files and file name is
<filename>_ddmmyyyy.csv
Ex: Jan-1-2022 shipment file is shipment_01012022.csv

You might also like