AI as a Service: Serverless machine learning with AWS

AI as a Service

Serverless machine learning with AWS

Peter Elger

Eóin Shanaghy

To comment go to liveBook

Manning

Shelter Island

For more information on this and other Manning titles go to

manning.com

Copyright

For online information and ordering of these and other Manning books, please visit manning.com. The publisher offers discounts on these books when ordered in quantity.

For more information, please contact

Special Sales Department

Manning Publications Co.

20 Baldwin Road

PO Box 761

Shelter Island, NY 11964

Email: [email protected]

©2020 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.

♾ Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.

ISBN: 9781617296154

dedication

For my parents, Noel and Kay—Eóin 

For my daughters Isobel and Katie, my parents Jacky and Julian, and my brother Jonathon—Peter

contents

foreword

preface

acknowledgments

about this book

about the authors

about the cover illustration

Part 1. First steps

1 A tale of two technologies

1.1  Cloud landscape

1.2  What is Serverless?

1.3  The need for speed

The early days

The Unix philosophy

Object orientation and patterns

Java, J2EE, .NET,

XML and SOAXML (Extensible Markup Language) SOA (service-oriented architecture)

Web speed

Cloud computing

Microservices (rediscovery)

Cloud native services

The trend: speed

1.4  What is AI?

History of AI

Real world AI

AI services

AI and machine learning

Deep learning

AI challenges

1.5  The democratization of compute power and artificial intelligence

1.6  Canonical AI as a Service architecture

Web application

Realtime services

Batch services

Communication services

Utility services

AI services

Data services

Operational support

Development support

Off-platform

1.7  Realization on Amazon Web Services

2 Building a serverless image recognition system, part 1

2.1  Our first system

2.2  Architecture

Web application

Synchronous services

Asynchronous services

Communication services

AI services

Data services

Development support and operational support

2.3  Getting ready

DNS domain and SSL/TLS certificate

Setup checklist

Get the code

Setting up cloud resources

2.4  Implementing the asynchronous services

Crawler service

3 Building a serverless image recognition system, part 2

3.1  Deploying the asynchronous services

Analysis service

3.2  Implementing the synchronous services

UI service

Front end service

3.3  Running the system

3.4  Removing the system

Part 2. Tools of the trade

4 Building and securing a web application the serverless way

4.1  The to-do list

4.2  Architecture

Web application

Synchronous services

Asynchronous services

Communication fabric

Utility services 85AI services

Data services

Development support and operational support

4.3  Getting ready

Getting the code

4.4  Step 1: The basic application

Resources

To-do service

Front end

Deploying step 1

4.5  Step 2: Securing with Cognito

Getting the code

User service

To-do service

Front-end service

Deploying step 2

5 Adding AI interfaces to a web application

5.1  Step 3: Adding a speech-to-text interface

Getting the code

Note service

Front-end updates

Deploying step 3

Testing step 3

5.2  Step 4: Adding text-to-speech

Getting the code

Schedule service

Front-end updates

Deploying step

Testing step

5.3  Step 5: Adding a conversational chatbot interface

Getting the code

Creating the bot

Front-end updates

Deploying step

Testing step

5.4  Removing the system

6 How to be effective with AI as a Service

6.1  Addressing the new challenges of Serverless

Benefits and challenges of Serverless

A production-grade serverless template

6.2  Establishing a project structure

The source repository--monorepo or polyrepo

Project folder structure

Get the code

6.3  Continuous deployment

Continuous deployment design

Implementing continuous deployment with AWS services

6.4  Observability and monitoring

6.5  Logs

Writing structured logs

Inspecting log output

Searching logs using CloudWatch Logs Insights

6.6  Monitoring service and application metrics

Service metrics

Application metrics

Using metrics to create alarms

6.7  Using traces to make sense of distributed applications

Enabling X-Ray tracing

Exploring traces and maps

Advanced tracing with annotations and custom metrics

7 Applying AI to existing platforms

7.1  Integration patterns for serverless AI

Pattern 1: Synchronous API

Pattern 2: Asynchronous API

Pattern 3: VPN Stream In

Pattern 4 VPN: Fully connected streaming

Which pattern?

7.2  Improving identity verification with Textract

Get the code

Text Analysis API

Client code

Deploy the API

Test the API

Remove the API

7.3  An AI-enabled data processing pipeline with Kinesis

Get the code

Deploying the API

7.4  On-the-fly translation with Translate

7.5  Testing the pipeline

7.6  Sentiment analysis with Comprehend

7.7  Training a custom document classifier

Create a training bucket

Upload training data

Create an IAM role

Run training

7.8  Using the custom classifier

7.9  Testing the pipeline end to end

7.10  Removing the pipeline

7.11  Benefits of automation

Part 3. Bringing it all together

8 Gathering data at scale for real-world AI

8.1  Scenario: Finding events and speakers

Identifying data required

Sources of data

Preparing data for training

8.2  Gathering data from the web

8.3  Introduction to web crawling

Typical web crawler process

Web crawler architecture

Serverless web crawler architecture

8.4  Implementing an item store

Getting the code

The item store bucket 219Deploying the item store

8.5  Creating a frontier to store and manage URLs

Getting the code

The frontier URL database

Creating the frontier API

Deploying and testing the frontier

8.6  Building the fetcher to retrieve and parse web pages

Configuring and controlling a headless browser

Capturing page output

Fetching multiple pages

Deploying and testing the fetcher

8.7  Determining the crawl space in a strategy service

8.8  Orchestrating the crawler with a scheduler

Grabbing the code

Using Step Functions

Deploying and testing the scheduler

9 Extracting value from large data sets with AI

9.1  Using AI to extract significant information from web pages

Understanding the problem

Extending the architecture

9.2  Understanding Comprehend’s entity recognition APIs

9.3  Preparing data for information extraction

Getting the code

Creating an S3 event notification

Implementing the preparation handler

Adding resilience with a dead letter queue (DLQ)

Creating the DLQ and retry handler

Deploying and testing the preparation service

9.4  Managing throughput with text batches

Getting the code

Retrieving batches of text for extraction

9.5  Asynchronous named entity abstraction

Get the code

Starting an entity recognition job

9.6  Checking entity recognition progress

9.7  Deploying and testing batch entity recognition

9.8  Persisting recognition results

9.9  Tying it all together

Orchestrating entity extraction

End-to-end data extraction testing

Viewing conference data extraction results

9.9  Wrapping up

Appendixes:

appendix A.  AWS account setup and configuration

appendix B.  Data requirements for AWS managed AI services

appendix C.  Data sources for AI applications

appendix D.  Setting up a DNS domain and certificate

appendix E.  Serverless Framework under the hood

index

front matter

foreword

For the past two decades, AI has played an increasingly significant role in our lives. It has done so quietly behind the scenes, as AI technologies have been employed by companies around the world to improve search results, product recommendations, and advertising, and even to assist healthcare workers to provide a better diagnosis. AI technologies are all around us, and soon, we’ll all travel in cars that drive themselves!

With this rise in prominence came a rise in demand for relevant skills. Engineers with expertise in machine learning or deep learning are often hoovered up by the big tech companies at huge salaries. Meanwhile, every application on the surface of the earth wants to use AI to improve its user experience. But the ability to hire the relevant skillsets and acquire the necessary volume of data to train these AI models remains a significant barrier to entry.

Fortunately, cloud providers are offering more and more AI services that remove the need for you to steep yourself in the art of collecting and cleaning up data and training AI models. AWS, for instance, lets you use the same technologies that power product recommendations for Amazon.com through Amazon Personalize, or the speech recognition technology that powers Alexa with Amazon Transcribe. Other cloud providers (GCP, Azure, IBM, and so on) also offer similar services, and it will be through these services that we will see AI-powered features in everyday applications. And as these services become better and more accessible, there will be less need for people to train their own AI models, except for more specialised workloads.

It’s great to finally see a book that focuses on leveraging these AI services rather than the nitty-gritty details of training AI models. This book explains the important concepts in AI and machine learning in layman’s terms, and describes them for exactly what they are, without all the hype and hyperbole that often accompany AI-related conversations. And the beauty of this book is that it is way more than how to use these AI services from AWS, but also how to build applications the serverless way. It covers everything from project organization, to continuous deployment, all the way to effective logging strategies and how to monitor your application using both service and application metrics. The later chapters of the book are also a treasure trove of integration patterns and real-world examples of how to sprinkle some AI magic into an existing application.

Serverless is a state of mind, a way of thinking about software development that puts the needs of the business and its customers at the forefront, and aims to create maximum business value with minimum effort by leveraging as many managed services as possible. This way of thinking leads to increased developer productivity and feature velocity, and often results in more scalable, resilient, and secure applications by building on the shoulders of giants such as AWS.

Serverless is not the future of how we build businesses around software; it’s the present and now, and it’s here to stay. This book will help you get started with Serverless development and show you how to integrate AI services into a Serverless application to enhance its user experience. Talk about hitting two birds with one stone!

Yan Cui

AWS Serverless Hero

Independent Consultant

preface

The fourth industrial revolution is upon us! The coming decade will likely see huge advances in areas such as gene editing, quantum computing, and, of course, artificial intelligence (AI). Most of us already interact with AI technology on a daily basis. This doesn’t just mean self-driving cars or automated lawn mowers. AI is far more pervasive than these obvious examples. Consider the product recommendation that Amazon just made when you visited their site, the online chat conversation you just had with your airline to re-book a flight, or the text that your bank just sent you warning of a possibly fraudulent transaction on your account. All of these examples are driven by AI and machine learning technology.

Increasingly, developers will be required to add smart AI-enabled features and interfaces to the products and platforms that they build. Early adopters of AI and machine learning have been doing this for some time, of course; however, this required a large investment in research and development, typically requiring a team of data scientists to train, test, deploy, and operate custom AI models. This picture is changing rapidly due to the powerful force of commoditization.

In his 2010 bestselling book, The Big Switch, Nicholas Carr compared cloud computing to electricity, predicting that eventually we would consume computing resources as a utility. Though we are not quite at the point of true utility computing, it is becoming clearer that this consumption model is fast becoming a reality.

You can see this in the explosive growth in the range and capability of cloud-native services. Commoditization of the cloud stack has given rise to the serverless computing paradigm. It is our belief that serverless computing will become the de facto standard architecture for building software platforms and products in the future.

In conjunction with the commoditization of the wider application stack, AI is also rapidly becoming a commodity. Witness the number of AI services that are available from the major cloud providers in areas such as image recognition, natural language processing, and chatbot interfaces. These AI services grow in number and capability month by month.

At our company, fourTheorem, we use these technologies on a daily basis to help our clients extend and improve their existing systems through the application of AI services. We help our clients to adopt serverless architectures and tools to accelerate their platform development efforts, and we use our experience to help restructure legacy systems so that they can run more efficiently on cloud.

It is the rapid growth and commoditization of these two technologies, Serverless and AI services, along with our experience of applying them to real-world projects, that led us to write this book. We wanted to provide an engineer’s guide to help you succeed with AI as a Service, and we wish you luck as you begin to master this brave new world of software development!

acknowledgments

Ask any technical book author ,and they will tell you that completing a book takes a lot of time and effort. It also requires the fantastic support of others. We are incredibly grateful for the many people who made completing this book possible.

First we would like to thank our families for their support, understanding, and patience while we worked to complete the book. Eóin would like to thank his amazing wife, Keelin, for her unending patience, moral support, and indispensable technical reviews. He would also like to thank Aoife and Cormac for being the best children in the world. Peter would like to thank his daughters, Isobel and Katie, just for being awesome.

Eóin and Peter would like to thank fourTheorem co-founder Fiona McKenna for her belief in this book, and her constant support and expertise in so many areas. We could not have done it without you.

Starting a project like this is the hardest part, and we are grateful for the people who helped in the beginning. Johannes Ahlmann contributed ideas, writing, and discussion that helped to shape what this book became. James Dadd and Robert Paulus provided invaluable support and feedback.

We would also like to thank the awesome team at Manning for making this book possible. In particular, we want to thank Lesley Trites, our development editor, for her patience and support. We would also like to thank Palak Mathur and Al Krinker, our technical development editors, for their review and feedback. Thank you to our project editor, Deirdre Hiam; Ben Berg, our copyeditor; Melody Dolab, our proofreader, and Ivan Martinović, our reviewing editor.

We would like to thank Yan Cui for writing the foreword to this book. Yan is an outstanding architect and champion of all things serverless, and we are grateful for his endorsement.

A big thanks to all of the reviewers for their feedback and suggestions for improvement to the text and examples: Alain Couniot, Alex Gascon, Andrew Hamor, Dwight Barry, Earl B. Bingham, Eros Pedrini, Greg Andress, Guillaume Alleon, Leemay Nassery, Manu Sareena, Maria Gemini, Matt Welke, Michael Jensen, Mykhaylo Rubezhanskyy, Nirupam Sharma, Philippe Vialatte, Polina Keselman, Rob Pacheco, Roger M. Meli, Sowmya Vajjala, Yvon Vieville,

A special thanks to Guillaume Alleon, technical proofreader, for his careful review and testing of the code examples.

Finally we wish to acknowledge the broader open source community, of which we are proud to participate in. We truly do stand on the shoulders of giants!

about this book

AI as a Service was written as an engineer’s guide to building AI-enabled platforms and services. The aim of the book is to get you up and running, and able to produce results quickly, without getting stuck in the weeds. AI and machine learning are big topics, and there is an awful lot to learn if you wish to master these disciplines. It is not our intent to discourage anyone from doing this, however if you need to get results quickly, this book will help you get up to speed.

The book examines two growing and increasingly important technologies: serverless computing and artificial intelligence. We examine these from a developer’s perspective to provide a practical, hands-on guide.

All of the major cloud vendors are engaged in a race to provide relevant AI services, such as

Image recognition

Speech-to-text, text-to-speech

Chatbots

Language translation

Natural language processing

Recommendations

This list will only expand over time!

The good news is that you do not need to be an AI or machine learning expert to use these offerings. This book will guide you in applying these services in your day-to-day work as a developer.

In tandem with the grown of AI services, it is now possible to build and deploy applications with a minimum of operational overhead using the serverless approach. Our belief is that within the next few years, the tools, techniques, and architectures described in this book will become part of the standard toolkit for enterprise platform development. This book will bring you up to speed quickly, and help you build new systems using serverless architectures and to apply AI services to your existing platforms.

Who should read this book

AI as a Service was written for full stack and back-end developers who are tasked with implementing AI-enhanced platforms and services. The book will also be of value to solution architects and product owners looking to understand how their systems can be augmented and improved with AI. DevOps professionals will gain valuable insights into the serverless way of building and deploying systems.

How this book is organized: a roadmap

This book is broken down into three sections covering nine chapters.

Part 1 provides some background and examines a simple serverless AI system:

Chapter 1 discusses the rise of serverless computing over the last few years, explaining why Serverless represents true utility cloud computing. Following this, it provides a brief overview of AI in order to bring readers with no experience with the topic up to speed.

Chapters 2 and 3 rapidly construct a serverless AI system that uses off-the-shelf image recognition technology. Readers can deploy and experiment with this system to explore how image recognition can be used.

Part 2 goes much deeper into the individual tools and techniques that developers need to know to become effective with serverless and off-the-shelf AI:

Chapter 4 examines how to build and deploy a simple serverless web application and then, perhaps more importantly, how to secure the application the serverless way.

Chapter 5 explores how we can add AI-driven interfaces to a serverless web application, including speech-to-text, text-to-speech, and a conversational chatbot interface.

Chapter 6 provides some specific advice on how to be an effective developer with this new technology set, including project structure, CI/CD, and observability--things that most developers getting to grips with this technology will need in their tool chest.

Chapter 7 looks in detail at how serverless AI can be applied to existing or legacy platforms. Here we provide advice on the generic patterns that can be applied, and also look at some point solutions to illustrate application of these patterns.

Part 3 looks at how we can bring together what you’ve learned from the first two parts in the context of a full-scale AI-driven system:

Chapter 8 examines gathering data at scale, using the example of a serverless web crawler.

Chapter 9 looks at how we can extract value from large data sets using AI as a Service, using the data collected from the serverless web crawler.

Readers should review the material in chapter 1 to get a basic grounding of the subject matter, and pay close attention to the content in chapter 2, where we describe how to set up a development environment. The book is best read in order, as each chapter builds on the examples and learning from the previous one.

About the code

This book contains many examples of source code, both in numbered listings and inline with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. Sometimes code is also in bold to highlight code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code.

In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts. Source code for the examples in this book is available for download from the publisher’s website.

All of the source code for this book is available at this repository: https:// github.com/fourTheorem/ai-as-a-service.

liveBook discussion forum

Purchase of AI as a Service includes free access to a private web forum run by Manning Publications where you can make comments about the book, ask technical questions, and receive help from the author and from other users. To access the forum, go to https://livebook.manning.com/#!/book/ai-as-a-service/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook .manning.com/#!/discussion.

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.

about the authors

about the cover illustration

The figure on the cover of AI as a Service is captioned Homme de la Forêt Noire, or The man from the Black Forest. The illustration is taken from a collection of dress costumes from various countries by Jacques Grasset de Saint-Sauveur (1757-1810), titled Costumes civils actuels de tous les peuples connus, published in France in 1788. Each illustration is finely drawn and colored by hand. The rich variety of Grasset de Saint-Sauveur’s collection reminds us vividly of how culturally apart the world’s towns and regions were just 200 years ago. Isolated from each other, people spoke different dialects and languages. In the streets or in the countryside, it was easy to identify where they lived and what their trade or station in life was just by their dress.

The way we dress has changed since then and the diversity by region, so rich at the time, has faded away. It is now hard to tell apart the inhabitants of different continents, let alone different towns, regions, or countries. Perhaps we have traded cultural diversity for a more varied personal life--certainly for a more varied and fast-paced technological life.

At a time when it is hard to tell one computer book from another, Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional life of two centuries ago, brought back to life by Grasset de Saint-Sauveur’s pictures.

Part 1. First steps

In part I we provide the ground work to get you up to speed on AI as a Service. In chapter 1 we look at the development and history of artificial intelligence and serverless computing. We review the current state of the art, and we categorize the available services on AWS into a standard architectural structure. In chapters 2 and 3 we dive right in and build a serverless image recognition system as our first AI as a Service platform.

1 A tale of two technologies

This chapter covers

Cloud landscape

What is Serverless?

What is artificial intelligence?

The democratizing power of Moore’s Law

A canonical AI as a Service architecture

Canonical architecture on Amazon Web Services

Welcome to our book! In these pages we are going to explore two exploding technologies: serverless computing and artificial intelligence. We will do this from an engineering perspective. When we say an engineering perspective, we mean that this book will provide you with a practical hands-on guide to get you up and running with AI as a Service, without getting bogged down in a lot of theory.

We imagine that like