Scribd
Upload
46 views26 pages

AWS API Gateway and S3 Integration (Encouraging The Correct Way) - by Sayed Imran - Medium

The document discusses how to directly upload files to an Amazon S3 bucket using the Amazon API Gateway service, without using SDKs or the AWS CLI. It describes creating an S3 bucket, an IAM role for API Gateway, and an API with parameters for the bucket name, folder path, and filename. Testing involves calling the API endpoint with the file, bucket/path parameters, and API key to upload the file to S3.

Uploaded by

Prof Silva
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
46 views26 pages

AWS API Gateway and S3 Integration (Encouraging The Correct Way) - by Sayed Imran - Medium

The document discusses how to directly upload files to an Amazon S3 bucket using the Amazon API Gateway service, without using SDKs or the AWS CLI. It describes creating an S3 bucket, an IAM role for API Gateway, and an API with parameters for the bucket name, folder path, and filename. Testing involves calling the API endpoint with the file, bucket/path parameters, and API key to upload the file to S3.

Uploaded by

Prof Silva
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Open in app

Search

Get unlimited access to the best of Medium for less than $1/week. Become a member

AWS API Gateway and S3 Integration


(encouraging the correct way)
Sayed Imran · Follow
6 min read · Oct 21, 2023

Listen Share More

Amazon S3
Amazon Simple Storage Service (Amazon S3), often referred S3, is a highly scalable,
secure, and flexible cloud storage service provided by Amazon Web Services (AWS).
It was introduced in March 2006 and has since become one of the most widely used
object storage services in the world.

S3 is designed to store and retrieve data of any type or size, making it a fundamental
building block for many cloud-based applications and services.
API Gateway
Amazon API Gateway is a fully managed service provided by Amazon Web Services
(AWS) that allows you to create, publish, secure, and manage APIs for your
applications. It serves a central entry point for your APIs, enabling you to connect to
backend services or AWS resources while handling essential tasks such as request
routing, authentication, authorization, monitoring, and more.

Requirement
There may be a requirement, where you want to upload images to S3 using APIs and
not any SDK or AWS CLI, rather directly from APIs!

Common Mistakes

Image source: https://www.luminis.eu/blog/cloud-en/creating-a-simple-api-stub-with-api-gateway-and-s3/

I’ve seen people using AWS Lambda as the intermediate to receive files via API
Gateway and then upload to S3, which according to me isn’t the optimal solution,
even if the files are required to be processed before saving, it takes twice the effort
to upload the file. (API gateway -> Lambda -> S3), which may lead to unwanted
corrupted files in the destination.

Possible Solution
Instead the files can be directly uploaded to the S3 via API Gateway and then,
Lambda can perform processing over it and may store the result as and where/when
required.

The following steps needs to be performed to achieve the above:

1. Create an S3 bucket to store the files (with public access if required)


2. Create an IAM role for API Gateway with permissions to PutObject in S3.

3. Create an API Gateway with the above created role attached.

4. Create an API with dynamic path parameters to accept the following:

Bucket Name

Folder Path (Optional)

Filename

5. Modify API Gateway settings to accept binary objects in payload.

6. Deploying API Gateway via a stage

7. Create an usage plan, so as to attach API key to avoid unauthorized access to the
resources.

Getting started
1. Creating an S3 bucket.
1.1(Optional) Unchecking Block all public access (will allow public read access)
1.2 (Optional) Creating S3 Bucket Policy for the above created bucket to allow end
users to access it.

{
"Id": "Policy1696244459937",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1696244457380",
"Action": [
"s3:GetObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::api-gateway-s3-tutorial/*",
"Principal": "*"
}
]
}

2. Create an IAM role with permissions policy to PutObject to the above created S3
bucket

2.1 Creating IAM role


2.2 Creating permission policy to attach to the above created role.

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::api-gateway-s3-tutorial/*"
}
]
}

Summary of IAM role

3. Creating API Gateway.

3.1 Creating API Gateway as REST API:

Summary for API Gateway Creation


4. Creating API Gateway Resources and Method to accept required parameters to
upload object to S3.

4.1 Creating resource {bucket} to accept bucket name in path.


4.2 Creating sub resource {filename} for the above resource to accept the file name
from path parameter.

4.3 Creating a PUT Method to the above created resource to upload files and
attaching the API Gateway Role created earlier.
4.3 Under Method Request for the above created method, check the API key required
field.

4.4 Under Integration Request for the above created method, edit the path
parameters as the following to accept bucket name and file name from the path
parameters.

5. Modifying API Gateway settings to accept any desired type of file content that we
want to upload. (.jpg and .png in our case).

6. Deploying the API with the above made configurations in a new stage.
7. Creating usage plan and API key to add the above created stage.

7.1 Creating API key


7.2 Creating usage plan and associating the previously created stage for API
Gateway.
7.3 Associating the API key
Now, the integration process is complete with API key authorization! To test the
setup we shall use Postman.

NOTE: Just to be sure, re-deploy the API gateway after making all the changes.

Testing the setup


1. Copy the URI from the deployed stage for API gateway and add the path for the
bucket name (along side folder path, if needed) and name of the file, attach the
file to be uploaded to the bucket as binary type in the payload, as shown below:

1.1 (Optional) To provide a destination folder path in s3 for a file, you need to use
“%2f” instead of “/”. For example if your path is:

api-gateway-s3-tutorial/folder1/wew.png, where

api-gateway-s3-tutorial is your bucket name,

folder1 is your folder and wew.png is your filename,


then the URI to upload shall be:

https://t8mhsswxx6.execute-api.ap-south-1.amazonaws.com/dev/api-gateway-s3-
tutorial%2ffolder1/wew.png

2. Add the API key in the header, as x-api-key as the key:

3. As soon as we click on Send, we receive a 200 OK response with empty response


body, implies the request was successful. And when we check for the same in the S3
console. We can see that the file has been uploaded!! (to the path if provided or else
in the root path of bucket).

Hence marks the completion of this tutorial, hope you find this small tutorial
insightful and would have solved your use case.

#aws #apigateway #s3 #cloud #lambda #automation #righteducation #devops


#knowledge #information #community #contribution
AWS Aws S3 Cloud Automation Api Gateway

Follow

Written by Sayed Imran


106 Followers

Multi Cloud Certified Associate | AWS | Google Cloud | Azure | Cloud and DevOps Enthusiast |

More from Sayed Imran

Sayed Imran in Google Cloud - Community

Terraform Remote Exec on Google Compute Engine VM Instance


In this article I’ve demonstrated a simple straight forward way to remotely execute scripts on a
Linux based VM Instance on Google Compute…
4 min read · Feb 24, 2023

122 3

Sayed Imran

Empowering Kubernetes Operator Development with Kopf: Introduction


to Operators (Part-1)
In recent years, Kubernetes has become the de facto standard for container orchestration,
enabling developers to manage containerized…

4 min read · 4 days ago

85
Sayed Imran in Knowledge Lens: A Rockwell Automation Company

Istio External Authorization with FastAPI (Python) Service.


In the world of microservices and distributed systems, maintaining robust security measures is
a top priority. As applications become more…

4 min read · Jul 14, 2023

116

Sayed Imran in Google Cloud - Community


What it takes to be Google Cloud Certified Associate Cloud Engineer
(ACE)
It fills me with immense pleasure to let you all know that, I’ve been certified by Google Cloud as
Associate Cloud Engineer on 13th March…

3 min read · Apr 10, 2023

72 1

See all from Sayed Imran

Recommended from Medium

Apurv Sheth in Towards AWS

The Most Popular File Upload to AWS S3 Using AWS Api Gateway
Introduction

6 min read · Aug 22, 2023

22
Akhilesh Mishra

You should stop writing Dockerfiles today — Do this instead


Using docker init to write Dockerfile and docker-compose configs

5 min read · Feb 8, 2024

2.4K 22

Lists

Coding & Development


11 stories · 453 saves

The New Chatbots: ChatGPT, Bard, and Beyond


12 stories · 307 saves

Natural Language Processing


1215 stories · 689 saves

Interesting Design Topics


226 stories · 374 saves
Lakindu Hewawasam in Bits and Pieces

Should You Use Amazon API Gateway in 2024? Consider Function URLs
Instead!
Did you know that you can now invoke Lambda functions using an HTTP API via Function
URLs?

8 min read · Feb 2, 2024

748 3
Artturi Jalli

I Built an App in 6 Hours that Makes $1,500/Mo


Copy my strategy!

· 3 min read · Jan 23, 2024

9.6K 126

Ajay-bj in Cloudnloud Tech Community

Creating an AWS S3 Bucket with Python: A Step-by-Step Guide


Today lets step by step instruction to create storage s3 using boto3

2 min read · Oct 27, 2023

21 1

Christopher Adamson

Building Scalable Microservices with AWS Lambda and Amazon API


Gateway
Microservices architecture has become popular for building scalable applications that are
made up of small, independent services that work…

6 min read · Oct 31, 2023

See more recommendations

You might also like