ChatGPT-like App Using Azure OpenAI

and Azure AI Search (Python)

Introduction
This solution demonstrates how to create a ChatGPT-like application using Azure OpenAI
and Azure AI Search services, with the backend written in Python. The project also includes
JavaScript, .NET, and Java samples. The application uses the Retrieval Augmented
Generation (RAG) pattern to access a GPT model (gpt-35-turbo) and retrieve relevant data
using Azure AI Search.

Features
The key features of this application include:

• Chat (multi-turn) and Q&A (single-turn) interfaces

• Rendering citations and thought processes for each answer
• UI settings to tweak behavior and experiment with options
• Integration with Azure AI Search for document indexing and retrieval
• Optional usage of GPT-4 with vision to analyze image-heavy documents
• Optional speech input/output for accessibility
• Optional automation of user login and data access via Microsoft Entra
• Performance tracing and monitoring with Application Insights
Architecture Diagram :

Azure Account Requirements

To deploy and run this example, the following are required:

• An Azure account (new users can get free Azure credits)

• Azure subscription with access to Azure OpenAI service
• Required Azure account permissions (e.g., Role Based Access Control Administrator,
User Access Administrator, or Owner)
• Permissions for Microsoft.Authorization/roleAssignments/write and
Microsoft.Resources/deployments/write at the subscription level

Cost Estimation
Costs vary depending on usage and region. The Azure pricing calculator can estimate costs
for the following resources:

• Azure App Service: Basic Tier with 1 CPU core, 1.75 GB RAM
• Azure OpenAI: Standard tier, GPT and Ada models (priced per 1K tokens used)
• Azure AI Document Intelligence: SO tier using pre-built layout (priced per document
page)
• Azure AI Search: Basic tier, 1 replica, free level of semantic search
• Azure Blob Storage: Standard tier with ZRS (Zone-redundant storage)
• Azure Monitor: Pay-as-you-go tier (costs based on data ingested)
To reduce costs, consider using free SKUs for various services. Be sure to take down the app
if it is no longer in use to avoid unnecessary expenses.

Getting Started
The project can be set up in multiple environments: GitHub Codespaces, VS Code Dev
Containers, or a local environment. The easiest method is GitHub Codespaces, which sets up
all the necessary tools in a web-based VS Code environment.

GitHub Codespace
You can run this repo virtually by using GitHub Codespaces, which will open a web-based
VS Code in your browser:

• Open GitHub Codespace

Once the codespace opens (this may take several minutes), open a terminal window

VS Code Dev Containers

1. A related option is VS Code Dev Containers, which will open the project in your local
VS Code using the Dev Containers Extensions
2. Open the Project:
Dev Container
3. In the VS Code window that opens, once the project files show up (this may take
several minutes), open a terminal window.

Local environment

1. Install the required tools:

• Azure Developer CLI
• Python 3.9 or 3.10 or 3.11
Important: Python and the pip package manager must be in the path
in Windows for the setup scripts to work.
Important: Ensure you can run python –version from console. On
Ubuntu, you might need to run sudo apt install python-is-python3 to
link python to python3.
 • Node.js
• Git
• Powershell 7 (pwsh)- For Windows users only.
Important: Ensure you can run pwsh.exe from a PowerShell terminal.
If this fails, you likely need to upgrade PowerShell.
2. Create a new folder and switch to it in the terminal.
3. Run this command to download the project code:

Azd init -t azure-search-openai-demo

Note that this command will initialize a git repository, so you do not need
to clone this repository.

Deployment
Follow these steps to provision Azure resources and deploy the application:

1. Login to your Azure account using `azd auth login`

2. Create a new azd environment using `azd env new`
3. Run `azd up` to provision resources and deploy the sample application
4. After successful deployment, access the app using the printed URL

Clean Up
To delete all resources created by this sample, run the command `azd down` and confirm
deletion when prompted.
Deploy again
If you’ve only changed the backend/frontend code in the app folder, then you don’t need to re-
provision the Azure resources. You can just run:

Azd deploy

If you’ve changed the infrastructure files (infra folder or azure.yaml), then you’ll need to re-
provision the Azure resources. You can do that by running:

Azd up

Sharing environments

1. Install the Azure CLI

2. Run Azd init -t azure-search-openai-demo or clone this repository.
3. Run azd env refresh -e {environment
name} They willA need the azd environment name, subscription ID, and location to
run this command. You can find those values in your .azure/{env name}/.env file. This
will populate their azd environment’s .env file with all the settings needed to run the
app locally.

4. Set the environment variable AZURE_PRINCIPAL_ID either in that .env file or in the
active shell to their Azure ID, which they can get with az ad signed-in-user show.

5. Run ./scripts/roles.ps1 or .scripts/roles.sh to assign all of the necessary roles to the

user. If they do not have the necessary permission to create roles in the subscription,
then you may need to run this script for them. Once the script runs, they should be
able to run the app locally.

Running locally
You can only run locally after having successfully run the azd up command. If you
haven’t yet, follow the Deploying
Steps above.
1. Run azd auth login
2. Change dir to app
 3. Run ./start.ps1 or ./start.sh or run the “VS Code Task: Start App” to start the
project locally.
See more tips in the local development guide

Using the App

• In Azure: navigate to the Azure WebApp deployed by azd. The URL is printed out
when azd completes (as “Endpoint”), or you can find it in the Azure portal.
• Running locally: navigate to 127.0.0.1:50505

Once in the web app:

• Try different topics in chat or Q&A context. For chat, try follow up questions,
clarifications, ask to simplify or elaborate on answer, etc.
• Explore citations and sources
• Click on “settings” to try different options, tweak prompts, etc.

Clean up
To clean up all the resources created by this sample:

1. Run azd down

2. When asked if you are sure you want to continue, enter y
3. When asked if you want to permanently delete the resources, enter y
The resource group and all the resources will be deleted

Guidance
Besides the tips below, you can find extensive documentation in the docs folder.

Customizing the UI and data

Once you successfully deploy the app, you can start customizing it for your needs: changing the
text, tweaking the prompts, and replacing the data.app customization guide as well as the. app
ingestion guide for more details

Monitoring with Application Insights

By default, deployed apps use Application Insights for the tracing of each request, along with the
logging of errors.

To see the performance data, go to the Application Insights resource in your resource group,
click on the “Investigate -> Performance” blade and navigate to any HTTP request to see the
timing data. To inspect the performance of chat requests, use the “Drill into Samples” button to
see end-to-end traces of all the API calls made for any chat request:

You can also see chart summaries on a dashboard by running the following command:

Azd monitor

Productionizing
This sample is designed to be a starting point for your own production application, but you
should do a thorough review of the security and performance before deploying to production.
Read through our Productionization Guide or more details.

Troubleshooting
Here are the most common failure scenarios and solutions:
1. The subscription (AZURE_SUBSCRIPTION_ID) doesn’t have access to the Azure OpenAI
service. Please ensure AZURE_SUBSCRIPTION_ID matches the ID specified in the Open Ai
Access Request guide.
2. You’re attempting to create resources in regions not enabled for Azure OpenAI (e.g. East
US 2 instead of East US), or where the model you’re trying to use isn’t enabled. See
this matrix of model availability .
3. You’ve exceeded a quota, most often number of resources per region. See
this article on quotas and limits.
4. You’re getting “same resource name not allowed” conflicts. That’s likely because you’ve
run the sample multiple times and deleted the resources you’ve been creating each
time, but are forgetting to purge them. Azure keeps resources for 48 hours unless you
purge from soft delete. See this article on purging resources.
5. You see CERTIFICATE_VERIFY_FAILED when the prepdocs.py script runs. That’s typically
due to incorrect SSL certificates setup on your machine. Try the suggestions in this
StackOverflow Answer.
6. After running azd up and visiting the website, you see a ‘404 Not Found’ in the browser.
Wait 10 minutes and try again, as it might be still starting up. Then try running azd
deploy and wait again. If you still encounter errors with the deployed app, consult the
guide on debugging app service deployments.Please file an issue if the logs don’t help
you resolve the error.

Getting help
This is a sample built to demonstrate the capabilities of modern Generative AI apps and
how they can be built in Azure. For help with deploying this sample, please post in Git
Hub issues . If you’re a Microsoft employee, you can also post in our Teams channel.

This repository is supported by the maintainers, not by Microsoft Support, so please use
the support mechanisms described above, and we will do our best to help you out.