Serverless Image Handler
Serverless Image Handler
Ian Hartz
Garvit Singh
June 2017
Contents
Overview ................................................................................................................................... 3
Cost ........................................................................................................................................ 4
Architecture Overview........................................................................................................... 4
Implementation Considerations .............................................................................................. 5
Cross-Origin Resource Sharing (CORS) ............................................................................... 5
AWS CloudFormation Template .............................................................................................. 6
Automated Deployment ........................................................................................................... 6
Prerequisites.......................................................................................................................... 6
What We’ll Cover................................................................................................................... 6
Step 1. Launch the Stack ....................................................................................................... 7
Step 2. Create an Image URL ................................................................................................ 8
Security ..................................................................................................................................... 9
Demo User Interface ............................................................................................................. 9
Additional Resources................................................................................................................ 9
Appendix A: Using the Demo UI ............................................................................................ 10
Appendix B: List of Supported Filters..................................................................................... 11
Appendix C: Rewrite Feature ..................................................................................................12
Appendix D: Watermarking Feature .......................................................................................14
Appendix E: Safe URL ............................................................................................................. 15
Appendix F: URL Encoding for Watermarking ...................................................................... 17
Appendix G: Viewing the Metadata ........................................................................................ 17
Appendix H: OpenCV vs Amazon Rekognition...................................................................... 18
Appendix I: Using Lambda Environment Variables ...............................................................19
Appendix J: Customizing Thumbor Lambda package ........................................................... 20
Appendix K: Troubleshooting .................................................................................................21
Common Errors....................................................................................................................21
Malformed URL ................................................................................................................21
Page 2 of 25
Amazon Web Services – Serverless Image Handler January 2019
Overview
Many Amazon Web Services (AWS) customers use images on their websites and mobile
applications to drive user engagement. Websites with large image files can experience high
load times, so in order to ensure a great user experience across different devices, developers
often provide multiple versions of each image to accommodate different bandwidth and
layout constraints. This process can be difficult to manage and cause time delays, as it often
requires version control, increased storage and compute costs for file reprocessing, and
coordination with application teams and web developers to update image files.
To help customers provide a low-latency website response, and decrease the cost of image
optimization, manipulation, and processing, AWS offers the Serverless Image Handler, a
solution that combines highly available, trusted AWS services and the open source image
processing suite Thumbor to enable fast and cost-effective image manipulation on the AWS
Cloud. This reference implementation automatically deploys and configures a serverless
architecture that is optimized for dynamic image manipulation, and that features Amazon
CloudFront for global content delivery and Amazon Simple Storage Service (Amazon S3) for
reliable and durable cloud storage at a low cost.
Page 3 of 25
Amazon Web Services – Serverless Image Handler January 2019
Cost
You are responsible for the cost of the AWS services used while running this solution. As of
the date of publication, the estimated cost for running the Serverless Image Handler for 1
million images processed, 15 GB storage, and 50 GB data transfer, with default settings in the
US East (N. Virginia) Region is as shown in the table below. This includes estimated charges
for Amazon API Gateway, AWS Lambda, Amazon CloudFront, and Amazon S3 storage.
Amazon S3 $0.23
This cost estimate does not account for Amazon S3 PUT and GET requests, which can vary
per scenario because modified images are cached in CloudFront. There is no additional cost
for using Thumbor, which is an open source tool. Prices are subject to change. For full details,
see the pricing webpage for each AWS service you will be using in this solution.
Architecture Overview
Automatic deployment of this solution configures the following components and
functionality.
Page 4 of 25
Amazon Web Services – Serverless Image Handler January 2019
Note: This solution is intended for customers with public applications who want to
provide an option for dynamically changing or manipulating their public images. As
a result, this template creates a publicly accessible, unauthenticated Amazon
CloudFront distribution and Amazon API Gateway endpoint in your account,
allowing anyone to access it. For more information on Amazon API Gateway
authorization, see the Security section.
The solution generates a CloudFront domain name that gives access to objects in a customer-
specified bucket, as well as a partial URL. In your custom front-end application, you can
append the CloudFront URL with the image path and name to create a direct URL for each
image object. Your users can then manipulate the image dynamically by adding Thumbor
filters to the URL. Additionally, the solution deploys an optional demo user interface where
you can more easily generate filters and tasks, and create example URLs that you can use for
your images. The demo UI is deployed in an Amazon S3 bucket to allow customers to
immediately start manipulating images with a simple web interface. Amazon CloudFront is
used to restrict access to the solution’s website bucket contents. For a list of filters that the
solution supports, see Appendix B.
For customers who don’t want to add Thumbor filters into their URLs or who already have a
naming convention in place, this solution includes a rewrite feature. This feature allows you
to match existing URL patterns and apply Thumbor parameters automatically within your
API. For more information, see Appendix C.
Implementation Considerations
Cross-Origin Resource Sharing (CORS)
The solutions template contains two parameters: Enable CORS and CORS Origin that
allow you to enable CORS for your API resources. CORS defines a way for client web
applications that are loaded in one domain to interact with resources in a different domain.
With CORS support, you may make API requests from a domain other than the API’s domain.
If you would like to change your CORS configuration after deployment, you can enable or
disable CORS with the ENABLE_CORS(YES/NO) and AWS Lambda environment variable,
or change the origin domain with the CORS_ORIGIN variable.
Page 5 of 25
Amazon Web Services – Serverless Image Handler January 2019
Automated Deployment
Follow the step-by-step instructions in this section to configure and deploy the Serverless
Image Handler into your account.
Time to deploy: Approximately 25 minutes
Prerequisites
Before you launch the solution’s AWS CloudFormation template, you must specify an existing
Amazon Simple Storage Service (Amazon S3) bucket. Use this bucket to store the images you
want to manipulate. For lower latency, use an S3 bucket in the same AWS Region where you
launch your AWS CloudFormation template.
We recommend deploying the optional demo user interface when you first deploy the
solution to test the solution’s functionality. For more information, see Appendix A.
Page 6 of 25
Amazon Web Services – Serverless Image Handler January 2019
Note: You are responsible for the cost of the AWS services used while running this
solution. See the Cost section for more details. For full details, see the pricing
webpage for each AWS service you will be using in this solution.
Note: This solution uses the AWS Lambda and Amazon API Gateway services,
which are currently available in specific AWS Regions only. Therefore, you must
launch this solution in an AWS Region where Lambda and API Gateway are
available. 1
3. On the Select Template page, keep the default settings for Stack and Template
Source.
4. Under Parameters, review the parameters for the template and modify them as
necessary. This solution uses the following default values.
Parameter Default Description
Origin S3 Bucket <Requires- The S3 bucket that contains the images that you will manipulate
Input>
Origin S3 Bucket Region us-east-1 The region that contains the Origin S3 Bucket
Enable Cors No Will this API require Cross-Origin Resource Sharing (CORS)
support?
Cors Origin * This value will be returned by the API in the Access-Control-
Allow-Origin header. A star (*) value will support any origin. We
recommend specifying a specific origin (e.g.
http://example.domain) to restrict cross-site access to your API.
Lambda Log Retention 1 Number of days to retain Lambda log data in CloudWatch logs
Page 7 of 25
Amazon Web Services – Serverless Image Handler January 2019
Deploy UI Yes The demo UI that will be deployed to the Demo S3 bucket. For
more information see Appendix A.
UI Prefix serverless- The prefix for the files in the Origin S3 bucket. Note that only
image- one folder can be used.
handler-ui/
5. Choose Next.
6. On the Options page, choose Next.
7. On the Review page, review and confirm the settings. Be sure to check the box
acknowledging that the template will create AWS Identity and Access Management
(IAM) resources.
8. Choose Create to deploy the stack.
You can view the status of the stack in the AWS CloudFormation Console in the Status
column. You should see a status of CREATE_COMPLETE in approximately 25 minutes.
1. In the AWS CloudFormation stack Outputs tab, copy the Sample Request URL.
2. Past the URL into your browser and append it with a Thumbor filter, the path (if
applicable), and filename for an image in your S3 bucket.
Note: If the image name and extension in the URL doesn’t match the image name in
the S3 bucket, you will receive an error.
In your custom front-end application, you can append the CloudFront URL with the bucket
folder path and image name to create a URL for each image object (example:
http://cloudfront…/path/image01.jpg).
Note: If you choose to deploy the optional demo user interface, you can experiment
with all of the supported image manipulation features, preview the results, and
create example URLs that you can use for your other images. For more information,
see Appendix A.
Page 8 of 25
Amazon Web Services – Serverless Image Handler January 2019
Security
When you build systems on AWS infrastructure, security responsibilities are shared between
you and AWS. This shared model can reduce your operational burden as AWS operates,
manages, and controls the components from the host operating system and virtualization
layer down to the physical security of the facilities in which the services operate. For more
information about security on AWS, visit the AWS Security Center.
This solution creates Amazon CloudFront and Amazon API Gateway resources that are
publicly accessible. Be aware that while this is likely appropriate for publicly facing websites,
it may not be appropriate for all customer use cases for this solution. AWS offers several
different options for end-to-end security, such as AWS Identity and Access Management
(IAM), Amazon Cognito User Pools, AWS Certificate Manager, and Amazon CloudFront
signed URLs. For private image handling use cases, AWS recommends using signed URLs
with Amazon CloudFront and implementing an Amazon API Gateway custom authorizer
with Amazon CloudFront to secure your stack.
Additional Resources
AWS services
• AWS CloudFormation
• AWS Lambda
• Amazon CloudFront
• Amazon API Gateway
• Amazon S3
Page 9 of 25
Amazon Web Services – Serverless Image Handler January 2019
Use the following procedure to experiment with all of the supported image manipulation
features, preview the results, and create example URLs that you can use for all your other
images:
1. In the AWS CloudFormation stack Outputs tab, select the DemoUI URL.
2. Adjust the available filters to understand the solutions behavior.
Note: To use your existing images after testing the functionality with the demo UI,
you must update the AWS CloudFormation Stack and change the Deploy UI
template parameter to No. This automatically updates the solution to work with the
images in the Amazon S3 bucket you specified in the OriginS3Bucket template
parameter.
Page 10 of 25
Amazon Web Services – Serverless Image Handler January 2019
Filters process the image in the order they are specified. For detailed instructions and more
options, see the Thumbor documentation.
Blur /filters:blur(7)/
Brightness /filters:brightness(40)/
Contrast /filters:contrast(40)/
Convolution /filters:convolution(1;2;1;2;4;2;1;2;1,3,false)/
Equalize /filters:equalize()/
Format /filters:format(jpeg)/
Grayscale /filters:grayscale()/
Noise /filters:noise(40)/
Quality /filters:quality(40)/
Resize /fit-in/800x1000/
RGB /filters:rgb(20,-20,40)/
Rotate /filters:rotate(90)/
Page 11 of 25
Amazon Web Services – Serverless Image Handler January 2019
Watermark /filters:watermark(https://<imageurl,x,y,z>)
SmartCrop /smart
Note: If you have already deployed the Serverless Image Handler solution and want
to use the rewrite feature, you must redeploy the AWS CloudFormation stack.
In the AWS Lambda console, select the solution’s primary Lambda function and set the
REWRITE_ENABLED environment variable to YES. Then, in the
REWRITE_PATTERNS environment variable, add your rewrite rules. The list of rules
must be comma separated and wrapped in square brackets. Each rule must be wrapped in
parentheses and contain regular expressions composed of two strings: one for the match
condition and one for substitution. Note that the regex must be compatible with Python. For
example:
[(r'^/string_to_match$', r'rewritten_string')]
The solution checks to see if the rewrite feature is enabled before it sends the request to
Thumbor. If it is enabled, the rewrite feature parses any customer defined rules in order until
it finds a match, and uses the provided patterns to transform the image while preserving the
originally requested URL. If no match is found, the original unmodified image request is sent
to Thumbor.
• This example matches a string (‘-zoom’) in a requested image name in the /s/ folder, and
returns a 630x630 resized image of the original file located in the /s/ folder (notice that
the original file name is appended with the string ‘-zoom’). This example demonstrates
the ability to create identifiers in the requested URL that map to different image filters.
Page 12 of 25
Amazon Web Services – Serverless Image Handler January 2019
REWRITE_ENABLED Yes
REWRITE_PATTERNS [(r'^/([s])/([0-9a-f]+)-(zoom)\.([a-zA-Z0-9]+)$',
r'/fit-in/630x630/s/\2.\4')]
• This example builds on the previous example; however, it also matches requests from
the /p/ folder and rewrites the request to retrieve the original image from the S3 origin’s
/product/ folder.
This example would be implemented by modifying the following Lambda function environment
variables:
Page 13 of 25
Amazon Web Services – Serverless Image Handler January 2019
REWRITE_ENABLED Yes
REWRITE_PATTERNS [(r'^/([p])/([0-9a-f]+)-(zoom)\.([a-zA-Z0-9]+)$',
r'/fit-in/630x630/product/\2.\4')]
• This example shows how you can create a comma-separated list of patterns to specify
multiple rewrite rules. Notice how the REWRITE_PATTERNS environment variable
value begins and ends with brackets, and each rule is wrapped in parentheses.
Additionally, we recommended placing your most frequently used patterns at the
beginning of the list to increase rule processing performance.
[(r'^/([s])/([0-9a-f]+)-(zoom)\.([a-zA-Z0-9]+)$', r'/fit-
in/630x630/sample/\2.\4'),(r'^/([s])/([0-9a-f]+)-(product)\.([a-zA-
Z0-9]+)$', r'/fit-in/220x220/sample/\2.\4'),(r'^/([s])/([0-9a-f]+)-
(cart)\.([a-zA-Z0-9]+)$', r'/fit-
in/53x53/sample/\2.\4'),(r'^/([s])/([0-9a-f]+)-(catalog)\.([a-zA-Z0-
9]+)$', r'/fit-in/120x120/sample/\2.\4'),(r'^/([s])/([0-9a-f]+)-
(gallery)\.([a-zA-Z0-9]+)$', r'/fit-
in/33x33/sample/\2.\4'),(r'^/([s])/([0-9a-f]+)-(list)\.([a-zA-Z0-
9]+)$', r'/fit-in/92x92/sample/\2.\4')]
Page 14 of 25
Amazon Web Services – Serverless Image Handler January 2019
When end-users access the page and load the image, Thumbor generates an authentication
code for the image URLs and filters, using the SECURITY_KEY provided in the Thumbor
config file. If the hash in the request URL and the authentication codes match, Thumbor
processes the image. For more information, see Thumbor’s safe URL.
Note: You can only use one environment variable at a time. Use
ALLOW_UNSAFE_URL=true for allowing unsafe URLs, or use
SECURITY_KEY='mysecuritykey' when safe URLs are needed.
```
http_key='mysecuritykey' # security key provided to lambda env
variable
http_path='200x200/smart/sub-folder/myimage.jpg' # sample options
for myimage
hashed = hmac.new(str(http_key),str(http_path),sha1)
encoded = base64.b64encode(hashed.digest())
signed_path = encoded.replace('/','_').replace('+','-')
```
Page 15 of 25
Amazon Web Services – Serverless Image Handler January 2019
The above procedure implements the following image URL and filters with options:
210x210/smart/filters:watermark(https://d2n9bu90z1w8su.cloudfront.net/serverless-
image-handler-ui/img/aws-logo-watermark.png,75,0,0)/serverless-image-handler-
ui/img/multiface.jpg
hash: Wwjr74-R7GrVN0XPT-Aq_DGFOh8=
Following this procedure successfully implements a secured URL. You may rotate security
keys as per your requirements and as often as needed by updating the Lambda environment
variable values. To verify the implementation, change the Security Key value in the Demo
UI console, navigate to the Lambda logs and check for the following error:
Page 16 of 25
Amazon Web Services – Serverless Image Handler January 2019
def encoding_string(string):
"""
Encoding URL per RFC 3986.
"""
return(quote(string,safe=''))
Example:
Watermarking image URL:
http://secure.gravatar.com/avatar/a432db53989289d4f251c52c72ff2bd3?d=identicon&r=
g&s=14000
As per RFC 3986, the value to be provided to watermark filter should be:
https%3A%2F%2Ffanyv88.com%3A443%2Fhttp%2Fsecure.gravatar.com%2Favatar%2Fa432db53989289d4f251c52c72ff2bd3
%3Fd%3Didenticon%26r%3Dg%26s%3D14000
To view the metadata on all Thumbor operations, add /meta/ to all of the URLs (for
example: http://<cloudfront-id>.net/meta/300x300/left/top/smart/<s3-prefix-
path>/picture.png). Once the processing is finished, Thumbor will return a .json object
with the metadata on the image and the operations that would have been performed. For
more information, see Metadata Endpoint.
{
thumbor: {
source: {
url: "path/to/my/nice/image.jpg",
width: 800,
Page 17 of 25
Amazon Web Services – Serverless Image Handler January 2019
height: 600
},
operations: [
{
type: "crop",
left: 10,
top: 10,
right: 300,
bottom: 200
},
{
type: "resize",
width: 300,
height: 200
},
{ type: "flip_horizontally" },
{ type: "flip_vertically" }
]
}
}
To use OpenCV
DETECTORS = [
#'thumbor.detectors.face_detector'
'thumbor_rekognition'
#'thumbor.detectors.profile_detector'
#'thumbor.detectors.glasses_detector',
'thumbor.detectors.feature_detector',
]
DETECTORS = [
'thumbor.detectors.face_detector',
#'thumbor_rekognition',
#'thumbor.detectors.profile_detector',
Page 18 of 25
Amazon Web Services – Serverless Image Handler January 2019
#'thumbor.detectors.glasses_detector',
'thumbor.detectors.feature_detector',
]
Un-comment the needed detector and upload the thumbor.conf file. For more
information, see Appendix J.
Certain common values in the Thumbor configuration can be controlled using Lambda
environment variables such as:
• RESPECT_ORIENTATION
• ALLOW_UNSAFE_URL
• SECURITY_KEY
• SEND_ANONYMOUS_DATA
• LOG_LEVEL
• ENABLE_CORS
Page 19 of 25
Amazon Web Services – Serverless Image Handler January 2019
Note that you must create two publicly accessible Amazon S3 buckets: my-bucket-name,
and my-bucket-name-<aws-region>.
$ cd serverless-image-handler/deployment
Page 20 of 25
Amazon Web Services – Serverless Image Handler January 2019
6. Deploy the distribution to an Amazon S3 bucket in your account. Note that you must
have AWS Command Line Interface installed.
bash
aws s3 cp ./dist/ s3://$DIST_OUTPUT_BUCKET-[region_name]/serverless-
image-handler/$VERSION/ --recursive --exclude "*" --include "*.zip"
aws s3 cp ./dist/serverless-image-handler.template
s3://$TEMPLATE_OUTPUT_BUCKET/serverless-image-handler/$VERSION/
Note that the solution will verify the source code should is located in the my-bucket-name-
[region-name] bucket with the prefix: serverless-image-handler/my-
version/serverless-image-handler.zip
```bash
https://s3.amazonaws.com/my-bucket-name/serverless-image-handler/my-
version/serverless-image-handler.template
```
Appendix K: Troubleshooting
To troubleshoot known issues with the Serverless Image Handler solution, AWS
recommends customers set the AWS Lambda environment variable LOG_LEVEL to DEBUG.
This will enable you to see detailed error logs in Amazon CloudWatch. For more
information about modifying environment variables, see Appendix I.
Common Errors
Malformed URL
This error can result if the security key provided in user request and the one provided to
Thumbor config do not match.
Resolution
Verify that both security keys match. For more information, see Appendix E.
Page 21 of 25
Amazon Web Services – Serverless Image Handler January 2019
Resolution
Customize your Lambda package to save the resulting image in the Amazon S3 bucket.
Currently, the solution requires you to manually update your result storage settings. For
more information, see tc_aws result storage.
Resolution
Configure your thumbor.conf file with the specific Thumbor parameter and customize
your Lambda package. See allow_environment_variables() for more information.
Page 22 of 25
Amazon Web Services – Serverless Image Handler January 2019
Note that AWS will own the data gathered via this survey. Data collection will be subject
to the AWS Privacy Policy. To opt out of this feature, complete one of the following tasks:
"Send" : {
"AnonymousUsage" : { "Data" : "Yes" }
},
to
"Send" : {
"AnonymousUsage" : { "Data" : "No" }
},
OR
b) After the solution has been launched, find the serverless-image-handler function
in the Lambda console and set the SEND_ANONYMOUS_DATA environment
variable to No.
Page 23 of 25
Amazon Web Services – Serverless Image Handler January 2019
Source Code
You can visit our GitHub repository to download the templates and scripts for this solution,
and to share your customizations with others.
Document Revisions
Date Change In sections
August 2017 Solution updated to add the rewrite feature and the Cost; Architecture Overview;
optional deployment of a demo UI Appendix A; Appendix B
September 2018 Added information on watermarking, URL encoding, Appendix C; Appendix D; Appendix
debugging, and troubleshooting E; Appendix F; Appendix G;
Appendix H; Appendix I; Appendix J
January 2019 Added information about using the demo UI, safe Appendix A; Appendix E;
URLs, and customizing the Thumbor Lambda Appendix J
package
Page 24 of 25
Amazon Web Services – Serverless Image Handler January 2019
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Notices
This document is provided for informational purposes only. It represents AWS’s current product offerings
and practices as of the date of issue of this document, which are subject to change without notice. Customers
are responsible for making their own independent assessment of the information in this document and any
use of AWS’s products or services, each of which is provided “as is” without warranty of any kind, whether
express or implied. This document does not create any warranties, representations, contractual
commitments, conditions or assurances from AWS, its affiliates, suppliers or licensors. The responsibilities
and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of,
nor does it modify, any agreement between AWS and its customers.
The Serverless Image Handler solution is licensed under the terms of the Amazon Software License available
at https://aws.amazon.com/asl/.
Page 25 of 25