18/4/25, 19:17 Realtime API - OpenAI API

Realtime API Beta

Copy page

Build low-latency, multi-modal experiences with the Realtime API.

The OpenAI Realtime API enables low-latency, multimodal interactions including speech-to-
speech conversational experiences and real-time transcription.

This API works with natively multimodal models such as GPT-4o and GPT-4o mini, offering
capabilities such as real-time text and audio processing, function calling, and speech
generation, and with the latest transcription models GPT-4o Transcribe and GPT-4o mini
Transcribe.

Get started with the Realtime API

You can connect to the Realtime API in two ways:

Using WebRTC, which is ideal for client-side applications (for example, a web app)

Using WebSockets, which is great for server-to-server applications (from your backend or
if you're building a voice agent over phone for example)

Start by exploring examples and partner integrations below, or learn how to connect to the
Realtime API using the most relevant method for your use case below.

Example applications

Check out one of the example applications below to see the Realtime API in action.

Realtime Console
To get started quickly, download and configure the Realtime console demo. See events flowing
back and forth, and inspect their contents. Learn how to execute custom logic with function
calling.

Realtime Solar System demo

A demo of the Realtime API with the WebRTC integration, navigating the solar system through
voice thanks to function calling.

Twilio Integration Demo

A demo combining the Realtime API with Twilio to build an AI calling assistant.

Realtime API Agents Demo

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 1/9
18/4/25, 19:17 Realtime API - OpenAI API

A demonstration of handoffs between Realtime API voice agents with reasoning model validation.

Partner integrations

Check out these partner integrations, which use the Realtime API in frontend applications and
telephony use cases.

LiveKit integration guide

How to use the Realtime API with LiveKit's WebRTC infrastructure.

Twilio integration guide

Build Realtime apps using Twilio's powerful voice APIs.

Agora integration quickstart

How to integrate Agora's real-time audio communication capabilities with the Realtime API.

Pipecat integration guide

Create voice agents with OpenAI audio models and Pipecat orchestration framework.

Client-side tool calling

Built with Cloudflare Workers, an example application showcasing client-side tool calling. Also
check out the tutorial on YouTube.

Use cases
The most common use case for the Realtime API is to build a real-time, speech-to-speech,
conversational experience. This is great for building voice agents and other voice-enabled
applications.

The Realtime API can also be used independently for transcription and turn detection use
cases. A client can stream audio in and have Realtime API produce streaming transcripts
when speech is detected.

Both use-cases benefit from built-in voice activity detection (VAD) to automatically detect
when a user is done speaking. This can be helpful to seamlessly handle conversation turns, or
to analyze transcriptions one phrase at a time.

Learn more about these use cases in the dedicated guides.

Realtime Speech-to-Speech
Learn to use the Realtime API for streaming speech-to-speech conversations.

Realtime Transcription

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 2/9
18/4/25, 19:17 Realtime API - OpenAI API

Learn to use the Realtime API for transcription-only use cases.

Depending on your use case (conversation or transcription), you should initialize a session in
different ways. Use the switcher below to see the details for each case.

Connect with WebRTC

WebRTC is a powerful set of standard interfaces for building real-time applications. The
OpenAI Realtime API supports connecting to realtime models through a WebRTC peer
connection. Follow this guide to learn how to configure a WebRTC connection to the Realtime
API.

Overview

In scenarios where you would like to connect to a Realtime model from an insecure client over
the network (like a web browser), we recommend using the WebRTC connection method.
WebRTC is better equipped to handle variable connection states, and provides a number of
convenient APIs for capturing user audio inputs and playing remote audio streams from the
model.

Connecting to the Realtime API from the browser should be done with an ephemeral API key,
generated via the OpenAI REST API. The process for initializing a WebRTC connection is as
follows (assuming a web browser client):

1 A browser makes a request to a developer-controlled server to mint an ephemeral API key.

2 The developer's server uses a standard API key to request an ephemeral key from the
OpenAI REST API, and returns that new key to the browser. Note that ephemeral keys
currently expire one minute after being issued.
3 The browser uses the ephemeral key to authenticate a session directly with the OpenAI
Realtime API as a WebRTC peer connection.

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 3/9
18/4/25, 19:17 Realtime API - OpenAI API

While it is technically possible to use a standard API key to authenticate client-side WebRTC
sessions, this is a dangerous and insecure practice because it leaks your secret key. Standard API
keys grant access to your full OpenAI API account, and should only be used in secure server-side
environments. We recommend ephemeral keys in client-side applications whenever possible.

Connection details

Connecting via WebRTC requires the following connection information:

URL https://api.openai.com/v1/realtime

Query model
Parameters

Realtime model ID to connect to, like gpt-4o-realtime-preview-2024-12-17

Headers Authorization: Bearer EPHEMERAL_KEY

Substitute EPHEMERAL_KEY with an ephemeral API token - see below for details on how
to generate one.

The following example shows how to initialize a WebRTC session (including the data channel
to send and receive Realtime API events). It assumes you have already fetched an ephemeral
API token (example server code for this can be found in the next section).

1 async function init() {

2 // Get an ephemeral key from your server - see server code below
3 const tokenResponse = await fetch("/session");
4 const data = await tokenResponse.json();
5 const EPHEMERAL_KEY = data.client_secret.value;
6
7 // Create a peer connection
8 const pc = new RTCPeerConnection();
9
10 // Set up to play remote audio from the model
11 const audioEl = document.createElement("audio");
12 audioEl.autoplay = true;
13 pc.ontrack = e => audioEl.srcObject = e.streams[0];
14
15 // Add local audio track for microphone input in the browser
16 const ms = await navigator.mediaDevices.getUserMedia({
17 audio: true
18 });
19 pc.addTrack(ms.getTracks()[0]);
20
21 // Set up data channel for sending and receiving events

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 4/9
18/4/25, 19:17 Realtime API - OpenAI API

22 const dc = pc.createDataChannel("oai-events");
23 dc.addEventListener("message", (e) => {
24 // Realtime server events appear here!
25 console.log(e);
26 });
27
28 // Start the session using the Session Description Protocol (SDP)
29 const offer = await pc.createOffer();
30 await pc.setLocalDescription(offer);
31
32 const baseUrl = "https://api.openai.com/v1/realtime";
33 const model = "gpt-4o-realtime-preview-2024-12-17";
34 const sdpResponse = await fetch(`${baseUrl}?model=${model}`, {
35 method: "POST",
36 body: offer.sdp,
37 headers: {
38 Authorization: `Bearer ${EPHEMERAL_KEY}`,
39 "Content-Type": "application/sdp"
40 },
41 });
42
43 const answer = {
44 type: "answer",
45 sdp: await sdpResponse.text(),
46 };
47 await pc.setRemoteDescription(answer);
48 }
49
50 init();

The WebRTC APIs provide rich controls for handling media streams and input devices. For
more guidance on building user interfaces on top of WebRTC, refer to the docs on MDN.

Creating an ephemeral token

To create an ephemeral token to use on the client-side, you will need to build a small server-
side application (or integrate with an existing one) to make an OpenAI REST API request for an
ephemeral key. You will use a standard API key to authenticate this request on your backend
server.

Below is an example of a simple Node.js express server which mints an ephemeral API key
using the REST API:

1 import express from "express";

2
3 const app = express();
4
5 // An endpoint which would work with the client code above - it returns

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 5/9
18/4/25, 19:17 Realtime API - OpenAI API

6 // the contents of a REST API request to this protected endpoint

7 app.get("/session", async (req, res) => {
8 const r = await fetch("https://api.openai.com/v1/realtime/sessions", {
9 method: "POST",
10 headers: {
11 "Authorization": `Bearer ${process.env.OPENAI_API_KEY}`,
12 "Content-Type": "application/json",
13 },
14 body: JSON.stringify({
15 model: "gpt-4o-realtime-preview-2024-12-17",
16 voice: "verse",
17 }),
18 });
19 const data = await r.json();
20
21 // Send back the JSON we received from the OpenAI REST API
22 res.send(data);
23 });
24
25 app.listen(3000);

You can create a server endpoint like this one on any platform that can send and receive HTTP
requests. Just ensure that you only use standard OpenAI API keys on the server, not in the
browser.

Sending and receiving events

To learn how to send and receive events over the WebRTC data channel, refer to the Realtime
conversations guide.

Connect with WebSockets

WebSockets are a broadly supported API for realtime data transfer, and a great choice for
connecting to the OpenAI Realtime API in server-to-server applications. For browser and
mobile clients, we recommend connecting via WebRTC.

Overview

In a server-to-server integration with Realtime, your backend system will connect via
WebSocket directly to the Realtime API. You can use a standard API key to authenticate this
connection, since the token will only be available on your secure backend server.

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 6/9
18/4/25, 19:17 Realtime API - OpenAI API

WebSocket connections can also be authenticated with an ephemeral client token (as shown above
in the WebRTC section) if you choose to connect to the Realtime API via WebSocket on a client
device.

Standard OpenAI API tokens should only be used in secure server-side environments.

Connection details

Speech-to-Speech Transcription

Connecting via WebSocket requires the following connection information:

URL wss://api.openai.com/v1/realtime

Query model
Parameters
Realtime model ID to connect to, like gpt-4o-realtime-preview-2024-12-17

Headers Authorization: Bearer YOUR_API_KEY

Substitute YOUR_API_KEY with a standard API key on the server, or an ephemeral token
on insecure clients (note that WebRTC is recommended for this use case).

OpenAI-Beta: realtime=v1

This header is required during the beta period.

Below are several examples of using these connection details to initialize a WebSocket
connection to the Realtime API.

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 7/9
18/4/25, 19:17 Realtime API - OpenAI API

ws module (Node.js) websocket-client (Python) WebSocket (browsers)

Connect using the ws module (Node.js) javascript

1 import WebSocket from "ws";

2
3 const url = "wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-12-17";
4 const ws = new WebSocket(url, {
5 headers: {
6 "Authorization": "Bearer " + process.env.OPENAI_API_KEY,
7 "OpenAI-Beta": "realtime=v1",
8 },
9 });
10
11 ws.on("open", function open() {
12 console.log("Connected to server.");
13 });
14
15 ws.on("message", function incoming(message) {
16 console.log(JSON.parse(message.toString()));
17 });

 

Sending and receiving events

To learn how to send and receive events over Websockets, refer to the Realtime conversations
guide.

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 8/9
18/4/25, 19:17 Realtime API - OpenAI API

https://platform.openai.com/docs/guides/realtime#connect-with-webrtc 9/9