Loading Data from AWS S3 into Snowflake

Anil Patel
Snowpro Core Certified Trainer
 Load Data from AWS S3 to Snowflake

01 Create a Free AWS Account

02 Create an IAM Group

03 Create an IAM User

04 Create an IAM Role

05 Create AWS S3 Bucket and folders

06 Upload files into AWS S3 Bucket and folders

07 Create the Database ,Table, File Format and Internal Stage in Snowflake
 Loading Data from AWS to Snowflake

Create the Database

Create integration
1 5 object for external
stage

Describe the integration

Create the table Select the data from Customer table object to fetch
2 6 external_id to use in
AWS S3 configuration

Define the file format

Create external stage & List the stage
3 Create an internal stage 7
8
4
Copy the data into customer table from external stage
 Load Data from AWS S3 to Snowflake
Create a Free AWS Account https://aws.amazon.com/
01
Login into AWS website and create an free aws account , need Credit card
 Load Data from AWS S3 to Snowflake
Create a Free AWS Account https://aws.amazon.com/
01
Login into AWS website and create an free aws account , need Credit card
02 – Create an IAM Group
 02 – Create an IAM Group
2

1
3

6 4
 03 – Create an IAM User
1

3
 03 – Create an IAM User
Assign Group to User

Skip

4
 03 – Create an IAM User
Review the information then click on Create user

5
 03 – Create an IAM User
User has been created and download the csv file (contains credentials)

5
 04 – Create an IAM Role
IAM roles are a secure way to grant permissions to entities that you trust.

2
 04 – Create an IAM Role
AWS Account Number.

3
 04 – Create an IAM Role
Enter AWS Account Number and enter any temporary external ID then click Next Permissions
 04 – Create an IAM Role
Assign “AmazonS3FullAccess” policy to Role then click Next:Tags
 04 – Create an IAM Role
Enter Role Name and description then click Create role
 04 – Create an IAM Role
The Role “Snowflake_role” has been created, we will configure trust relationship later
05 – Create AWS S3 Bucket and folders
Go to S3 under services or search S3 in “search for services” box

Then Create bucket

05 – Create AWS S3 Bucket and folders
Enter the bucket name “snowflakes3aws” it should be unique name and select AWS Region and
uncheck “block all public access” and acknowledge
05 – Create AWS S3 Bucket and folders
Bucket snowflakes3aws has been created then create folders inside the buckets
 06 – Upload files into AWS S3 Bucket and folders
Drag and drop the customers.csv file into csv folder(under the snowflakes3aws bucket)
Data Loading from AWS S3 to Snowflake

Snowflake cloud data warehouse architecture is

specifically designed to work with the cloud such
as
Amazon AWS,
Google cloud platform
and Microsoft Azure.

Loading and Unloading tables, Snowflake use

different methods compared to others relational
database.

Snowflake uses staging area to store the file

that you can in your COPY command
There are 2 types of Stages in Snowflake

Internal stage - Stores the data files internally within Snowflake. Internal stages can be
either permanent or temporary.

External stage – References the data files stored in a location outside of Snowflake.
Currently, the following cloud storage services are supported:
• Amazon S3 buckets
• Google Cloud Storage buckets
• Microsoft Azure containers

The storage location can be either private/protected or public

 Loading Data from AWS to Snowflake

Create the Database

Create integration
1 5 object for external
stage

Describe the integration

Create the table Select the data from Customer table object to fetch
2 6 external_id to use in
AWS S3 configuration

Define the file format

Create external stage & List the stage
3 Create an internal stage 7
8
4
Copy the data into customer table from external stage
07 – Create the Database , Table, File Format and Internal Stage in Snowflake
*Need AccountAdmin Privilege
1.Create the Database
CREATE DATABASE aws_integration;

2. create the table customer table

CREATE TABLE CUSTOMER
(
FName varchar(80),
LName varchar(80),
Email varchar(100),
Date_Of_Birth DATE,
City varchar(100),
Country varchar(100)
);

3. Define the file format for customers.csv file ( delimiter - pipe)

CREATE FILE FORMAT PIPE_DELIMITER
TYPE = CSV
FIELD_DELIMITER = '|'
FIELD_OPTIONALLY_ENCLOSED_BY = '"'
SKIP_HEADER = 1
DATE_FORMAT = 'YYYY-MM-DD';

4. Create an internal stage

CREATE STAGE CUSTOMER_STAGE FILE_FORMAT = PIPE_DELIMITER;
07 – Create the Database , Table, File Format and Internal Stage in Snowflake
5.Create integration object for external stage
create or replace storage integration s3_integration
type = external_stage
storage_provider = s3
enabled = true
storage_aws_role_arn = 'arn:aws:iam::325530134896:role/snowflake_role'
storage_allowed_locations = ('s3://snowflakes3aws/csv/’);

6. Describe the integration object to fetch external_id to use in AWS S3

configuration
DESC INTEGRATION s3_integration;
4.1 – Configure Trust Relationships from Snowflake
The Role “Snowflake_role” has been created, we will configure trust relationship later
4.1 – Configure Trust Relationships from Snowflake
4.2 – Configure Trust Relationships from Snowflake
07 – Successfully created Trust relationship/Connection
between AWS and Snowflake
7. Create external stage
create or replace stage EXTERNAL_CSV_STAGE
URL = 's3://snowflakes3aws/csv/'
STORAGE_INTEGRATION = s3_integration
file_format = PIPE_DELIMITER;

8. List the stage

List @<stage_name>;
list @EXTERNAL_CSV_STAGE;

9. Copy the data into customer table from external

stage
copy into customer from @EXTERNAL_CSV_STAGE
on_error = CONTINUE;

10. Select the data from Customer table

select * from customer;
Thank You