Generative AI Application Integration Patterns: Integrate large language models into your applications

Generative AI Application Integration Patterns

Integrate large language models into your applications

Juan Pablo Bustos

Luis Lopez Soria

Generative AI Application Integration Patterns

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Senior Publishing Product Manager: Tushar Gupta

Acquisition Editor – Peer Reviews: Tejas Mhasvekar

Project Editor: Meenakshi Vijay

Content Development Editor: Shazeen Iqbal

Copy Editor: Safis Editing

Technical Editor: Gaurav Gavas

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Presentation Designer: Rajesh Shirsath

Developer Relations Marketing Executive: Maran Fernandes

First published: August 2024

Production reference: 1290824

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK.

ISBN 978-1-83588-760-8

www.packt.com

Foreword

The field of Artificial Intelligence is in the midst of bringing about a profound transformation in business. One of the pivotal areas of this transformation is in the integration with applications that are already running businesses, and adding value to them. This book, Generative AI Application Integration Patterns explores these recurrent themes as patterns of integration with generative AI. It serves as a timely guide for navigating the nuanced landscape of integrating GenAI into existing business applications. It peers into the fog of the immense potential of GenAI, and provides practical clarity that may help you revolutionize key competitive aspects of your business operations; areas like enhancing customer experiences to domains such as streamlining internal processes. By focusing on the practical aspects of integration, the authors equip readers with the background knowledge and tools they need to leverage this transformative technology even more effectively.

Juan and Luis delve into the underlying practical aspects of generative AI, and in doing so, provide a solid foundation for understanding and actualizing its capabilities and navigating its limitations. They explore the different architectural integration patterns that can be employed for more seamless integration, and consider how factors such as scalability, performance, and security should be taken into account. The practical case studies presented throughout the book showcase how successful implementations of GenAI can be realized across industries. These examples are exemplary blueprints that demonstrate how businesses can leverage this technology to achieve tangible outcomes.

In addition, this book addresses some of the critical considerations of responsible AI development and deployment. It emphasizes the importance of ethical considerations, data privacy, and bias mitigation, that help ensure that GenAI is utilized in a manner that aligns with ethical principles and societal values. This holistic approach helps readers not only gain technical expertise but also develop a deeper appreciation of the ethical challenges and implications of their work.

Generative AI Application Integration Patterns is a well-written, engaging and very relevant set of blueprints that technology and business leaders, as well as developers, should be aware of as they seek to integrate applications with the promise presented in GenAI. I encourage you to dive deep into the examples, reflect on the concepts presented in this book, and embark on the exciting journey of discovery and innovation in harnessing the potential of GenAI. The future of business is being shaped by AI, and this book is an essential companion on that path.

Dr. Ali Arsanjani

Director of Applied AI Engineering, Google

Contributors

About the authors

Juan Pablo Bustos is a seasoned technology professional specializing in artificial intelligence and machine learning. With a background in computer science, Juan has held leadership positions at major tech companies including Google, Stripe, and Amazon Web Services. His expertise spans AI services, solution architecture, and cloud computing. Juan is passionate about helping organizations leverage cutting-edge technologies to drive innovation and deliver value.

I’m deeply grateful to my wife Cinthia for her constant support, encouragement, and for being my sounding board for even my craziest ideas. Thanks to Penny and Andrew, my kids, for their patience. I’d like to acknowledge my father, Dr. Sergio Bustos, for encouraging me to pursue computer science. I’m indebted to Dr. Ali Arsanjani, Robert Love, and Todd Reagan for their invaluable mentorship and friendship. A special thanks to my friend Luis, my partner in crime for this book. Finally, I’d like to recognize Gemini, Claude, and ChatGPT for their invaluable help and for democratizing access to GenAI.

Luis Lopez Soria is an experienced software architect specializing in AI/ML. He has gained practical experience from top firms across heavily regulated industries like healthcare and finance, as well as big tech firms like AWS and Google. He brings a blended approach from his experience managing global partnerships, AI product development, and customer-facing roles. Luis is passionate about learning new technologies and using these to create business value.

I want to thank my parents and sister. Your unwavering support, willingness to lend an ear, and readiness to brainstorm ideas have been invaluable. To my grandfather Felix and uncle Ricardo: your presence and support by my side made this dream a reality.

A special thanks goes to Chris K. and Juan B., whose early influence on my career cannot be overstated. Your constant push for excellence and valuable input have left an indelible mark on my professional growth and, by extension, on this book.

To all of you, and the many others who have contributed in ways both big and small, I offer my heartfelt gratitude. This book stands as a testament to your belief in me and your ongoing support.

About the reviewer

Aditi Khare holds 8+ years of experience in the AI research and product engineering space.

She is passionate about AI research, open source, and building production-grade AI products. She has worked for Fortune 50 product companies. She has completed a big data analytics course at the Indian Institute of Management, Ahmedabad, and a master’s in computer applications at K. J. Somaiya Institute of Management, Mumbai. In her spare time, she enjoys reading AI-related research papers and publishing research paper summaries through her LinkedIn newsletter. For more information about her, visit https://www.linkedin.com/in/aditi-khare-5840977b/.

I’d like to dedicate my contribution to this book to the loving memory of my beloved mom, the late Mrs. Shashi Khare, who has always been my inspiration and the reason for my achievements.

I’d like to thank my father Mr. Alok Khare and my brother Ayush Khare for being very supportive and acting as a guiding force in all my achievements.

Join our community on Discord

Join our community’s Discord space for discussions with the authors and other readers:

https://packt.link/genpat

Contents

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Introduction to Generative AI Patterns

From AI predictions to generative AI

Predictive AI vs generative AI use case ideation

A change in the paradigm

Predictive AI use case development – simplified lifecycle

Generative AI use case development – simplified lifecycle

General generative AI concepts

Generative AI model architectures

Techniques available to optimize foundational models

Techniques to augment your foundational model responses

Constant evolution across the generative AI space

Introducing generative AI integration patterns

Summary

Identifying Generative AI Use Cases

When to consider generative AI

Realizing business value

Identifying Generative AI use cases

Potential business-focused use cases

Generative AI deployment and hosting options

Summary

Designing Patterns for Interacting with Generative AI

Defining an integration framework

Entry point

Prompt pre-processing

Inference

Results post-processing

Selecting from amongst multiple outputs

Refining generated outputs

Results presentation

Logging

Summary

Generative AI Batch and Real-Time Integration Patterns

Batch and real-time integration patterns

Different pipeline architectures

Application integration patterns in the integration framework

Entry point

Prompt pre-processing

Inference

Result post-processing

Result presentation

Use case example – search enhanced by GenAI

Batch integration – document ingestion

Real-time integration – search

Summary

Integration Pattern: Batch Metadata Extraction

Use case definition

Architecture

Entry point

Prompt pre-processing

Inference

Result post-processing

Result presentation

Summary

Integration Pattern: Batch Summarization

Use case definition

Architecture

Entry point

Prompt pre-processing

Inference

Result post-processing

Result presentation

Summary

Integration Pattern: Real-Time Intent Classification

Use case definition

Architecture

Entry point

Prompt pre-processing

Inference

Result post-processing

Result presentation

Logging and monitoring

Summary

Integration Pattern: Real-Time Retrieval Augmented Generation

Use case definition

Architecture

Entry point

Prompt pre-processing

Inference

Result post-processing

Result presentation

Use case demo

The Gradio app

Summary

Operationalizing Generative AI Integration Patterns

Operationalization framework

Data layer

A real-world example: Part 1

Training layer

A real-world example: Part 2

Inference layer

A real-world example: Part 3

Operations layer

CI/CD and MLOps

Monitoring and observability

Evaluation and monitoring

Alerting

Distributed tracing

Logging

Cost optimization

Summary

Embedding Responsible AI into Your GenAI Applications

Introduction to responsible AI

Fairness in GenAI applications

Interpretability and explainability

Privacy and data protection

Safety and security in GenAI systems

Google’s approach to responsible AI

Google’s Secure AI Framework (SAIF)

Google’s Red Teaming approach

Anthropic’s approach to responsible AI

Summary

Other Books You May Enjoy

Index

Landmarks

Cover

Index

Preface

More than five years ago, before the widespread adoption of generative AI, we were searching for new ways to enhance application development and user experiences. A few years later, we found ourselves deeply immersed in the world of generative AI, which has opened countless possibilities for innovation. Before discovering the transformative potential of this technology, we, the authors, explored various machine learning techniques, experimenting with different models and reading countless research papers. With generative AI, we found more than just a powerful tool; we discovered a new paradigm that is reshaping how we approach software development and problem-solving.

This book is about sharing the excitement and insights we have gained while exploring and implementing generative AI solutions. It is intended to guide you through the wide world of generative AI applications, with a focus on practical design patterns and real-world implementations. We will cover concepts ranging from basic to advanced, taking you on a journey like the one I experienced while learning to harness the power of generative AI.

Who this book is for

This book caters to a wide audience with a keen interest in generative AI and its practical applications:

Software developers and engineers with foundational knowledge of AI/ML and Python

Software architects seeking generative AI best practices and design patterns

Data scientists, researchers, and analysts looking to incorporate generative AI into their workflows

Technical product managers with a background in software development

AI enthusiasts who want to deepen their understanding of generative AI implementation strategies

What this book covers

Chapter 1, Introduction to Generative AI Patterns, provides an overview of generative AI concepts, architectures, and their potential impact on application development.

Chapter 2, Identifying Generative AI Use Cases, guides readers through the process of identifying and evaluating potential use cases for generative AI across various domains.

Chapter 3, Designing Patterns for Interacting with Generative AI, explores different strategies for effectively communicating with and leveraging generative AI models in applications.

Chapter 4, Generative AI Batch and Real-Time Integration Patterns, discusses the different approaches for integrating generative AI into both batch-processing and real-time systems.

Chapter 5, Integration Pattern: Batch Metadata Extraction, demonstrates how to implement generative AI to extract metadata from large datasets in batch mode.

Chapter 6, Integration Pattern: Batch Summarization, covers techniques for using generative AI to create summaries of large volumes of text data.

Chapter 7, Integration Pattern: Real-Time Intent Classification, shows how to implement generative AI to classify user intents in real-time applications.

Chapter 8, Integration Pattern: Real-Time Retrieval Augmented Generation, explores advanced techniques for building question-answering systems using generative AI and retrieval augmented generation.

Chapter 9, Operationalizing Generative AI Integration Patterns, provides guidance on deploying, monitoring, and maintaining generative AI systems in production environments.

Chapter 10, Embedding Responsible AI into Your GenAI Applications, addresses ethical considerations and best practices for responsible use of generative AI in applications.

To get the most out of this book

To fully benefit from this book, you should have:

A solid understanding of Python programming

Familiarity with basic machine learning concepts

Experience with software development and application architectures

Access to a development environment capable of running Python and installing necessary libraries

The chapters contain both theoretical explanations and practical code examples. To run the code in the book, you can follow these steps:

Clone the GitHub repository associated with this book.

Set up a Python environment with the required dependencies (listed in the repository).

Download or access the necessary generative AI models as instructed in each chapter.

Run the provided Jupyter notebooks or Python scripts.

Alternatively, you can use cloud-based platforms that offer pre-configured environments for AI development, such as Google Colab or Amazon SageMaker, to run the examples without setting up a local environment.

Download the example code files

The code bundle for the book is hosted on GitHub at https://github.com/PacktPublishing/Generative-AI-Integration-Patterns-1E. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://packt.link/gbp/9781835887608.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText

: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter/X handles. For example: "Mount the downloaded

WebStorm-10*.dmg

disk image file as another disk in your system."

A block of code is set as follows:

generation_config = {

max_output_tokens

:

8192

,

temperature

:

0

,

top_p

:

0.95

, }

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

responses = model.generate_content( [prompt],

generation_config=generation_config,

safety_settings=safety_settings,

stream=

False

, )

Any command-line input or output is written as follows:

#

cp

/usr/src/asterisk-addons/configs/cdr_mysql.conf.sample /etc/asterisk/cdr_mysql.conf

Bold: Indicates a new term, an important word, or words that you see on the screen. For instance, words in menus or dialog boxes appear in the text like this. For example: "Select System info from the Administration panel."

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email

[email protected]

and mention the book’s title in the subject of your message. If you have questions about any aspect of this book, please email us at

[email protected]

.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you reported this to us. Please visit http://www.packtpub.com/submit-errata, click Submit Errata, and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at

[email protected]

with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit http://authors.packtpub.com.

Share your thoughts

Once you’ve read Generative AI Application Integration Patterns, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.