Gateway Realtime

Realtime traffic uses the same Gateway policy layer as HTTP inference: model resolution, usage limits, guardrails, and trace capture all apply. Streaming recovery is limited because bytes are committed to the client as soon as they arrive.

When To Use It#

Use Gateway Realtime when:

• the application already uses OpenAI-compatible realtime clients;
• you want voice or multimodal sessions to pass through Provon routing and evidence capture;
• multiple provider keys or regions need the same application-level model ID.

Do not use Gateway Realtime for plain request/response chat; use /chat/completions instead.

Supported Paths#

Append these operation paths to the Gateway base URL:

Hosted base URL:

wss://gateway.provon.dev/v1/realtime?model=openai/gpt-4o-realtime-preview

Local development base URL:

ws://127.0.0.1:3000/gateway/v1/realtime?model=openai/gpt-4o-realtime-preview

The /v1 prefix is optional for the WebSocket path; /gateway/v1/realtime and /v1/realtime are both accepted.

Prerequisites#

1. The deployment runtime can terminate WebSocket upgrade requests. Cloudflare Workers support this natively. A standalone Node server may require a WebSocket-capable fronting layer.
2. The target Provider Key has a mapping that declares the realtime endpoint family and the model you request.
3. The project API key used by the client has gateway:invoke.

Query eligible realtime models:

export PROVON_API_URL="https://api.provon.dev/v1"

curl \
  "$PROVON_API_URL/gateway/model-selection?endpoint=realtime&features=audio-input,audio-output" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Connect From A Browser#

const ws = new WebSocket(
  'wss://gateway.provon.dev/v1/realtime?model=openai/gpt-4o-realtime-preview',
  [],
  {
    headers: {
      Authorization: `Bearer ${PROVON_API_KEY}`,
      'x-otel-gen-ai-conversation-id': 'realtime-session-123',
      'x-otel-user-id': 'user-456',
    },
  },
);

ws.addEventListener('open', () => {
  ws.send(
    JSON.stringify({
      type: 'session.update',
      session: { model: 'gpt-4o-realtime-preview' },
    }),
  );
});

ws.addEventListener('message', (event) => {
  const message = JSON.parse(event.data);
  console.log(message);
});

The model query parameter can be provider-qualified (openai/gpt-4o-realtime-preview) or a plain model ID if a project Model Policy resolves it. Provon strips the provider prefix before forwarding to OpenAI-compatible upstreams and adjusts the path for Azure targets.

Client Secret#

OpenAI's client SDKs often expect an ephemeral secret from /realtime/client_secrets. The Gateway normalizes the request body, resolves the model, and forwards it to the upstream provider:

curl -X POST "$PROVON_API_URL/gateway/v1/realtime/client_secrets" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-realtime-preview",
    "session": {
      "model": "gpt-4o-realtime-preview",
      "modalities": ["audio", "text"]
    }
  }'

Calls And Transcription Sessions#

/realtime/calls and /realtime/transcription_sessions follow the same normalization rules as /realtime/client_secrets. Send the OpenAI-compatible body; Provon resolves the nested model fields and routes to the selected target.

Fallback And Recovery#

Realtime WebSocket connections support ordered fallback across multiple eligible Provider Keys. If the first upstream WebSocket cannot be established, the Gateway tries the next prepared connection. Once the bridge is active, recovery is no longer possible without reconnecting the client.

Configure fallback in the Provider Key's fallbackPolicy as described in Routing and reliability.

Error Codes#

Limitations#

• Realtime uses raw WebSocket passthrough. The Gateway does not inspect or rewrite individual realtime events after the bridge is established.
• Request/response guardrails that act on the full body run only on the setup handshake, not on each streaming event.
• Trace capture records the session lifecycle and upstream attempt, not per-event telemetry.
• Node deployments require a runtime or fronting layer that can upgrade HTTP to WebSocket.