# Gateway Realtime

Provon AI Gateway supports the OpenAI Realtime API over WebSocket. The Gateway upgrades the client
connection, resolves the requested model to an eligible Provider Key, opens an upstream WebSocket,
and bridges the two sockets. The application keeps its existing `gateway:invoke` project API key;
the upstream provider credential never reaches the client.

Realtime traffic uses the same Gateway policy layer as HTTP inference: model resolution, usage
limits, guardrails, and trace capture all apply. Streaming recovery is limited because bytes are
committed to the client as soon as they arrive.

## When To Use It

Use Gateway Realtime when:

- the application already uses OpenAI-compatible realtime clients;
- you want voice or multimodal sessions to pass through Provon routing and evidence capture;
- multiple provider keys or regions need the same application-level model ID.

Do not use Gateway Realtime for plain request/response chat; use
[`/chat/completions`](./routing.md) instead.

## Supported Paths

Append these operation paths to the Gateway base URL:

| Operation                          | Path                               |
| ---------------------------------- | ---------------------------------- |
| Realtime WebSocket                 | `/realtime?model=<provider/model>` |
| Realtime client secret             | `/realtime/client_secrets`         |
| Realtime calls (OpenAI SDK helper) | `/realtime/calls`                  |
| Realtime transcription sessions    | `/realtime/transcription_sessions` |

Hosted base URL:

```text
wss://gateway.provon.dev/v1/realtime?model=openai/gpt-4o-realtime-preview
```

Local development base URL:

```text
ws://127.0.0.1:3000/gateway/v1/realtime?model=openai/gpt-4o-realtime-preview
```

The `/v1` prefix is optional for the WebSocket path; `/gateway/v1/realtime` and `/v1/realtime` are
both accepted.

## Prerequisites

1. The deployment runtime can terminate WebSocket upgrade requests. Cloudflare Workers support this
   natively. A standalone Node server may require a WebSocket-capable fronting layer.
2. The target Provider Key has a mapping that declares the `realtime` endpoint family and the model
   you request.
3. The project API key used by the client has `gateway:invoke`.

Query eligible realtime models:

```bash
export PROVON_API_URL="https://api.provon.dev/v1"

curl \
  "$PROVON_API_URL/gateway/model-selection?endpoint=realtime&features=audio-input,audio-output" \
  -H "Authorization: Bearer $PROVON_API_KEY"
```

## Connect From A Browser

```javascript
const ws = new WebSocket(
  'wss://gateway.provon.dev/v1/realtime?model=openai/gpt-4o-realtime-preview',
  [],
  {
    headers: {
      Authorization: `Bearer ${PROVON_API_KEY}`,
      'x-otel-gen-ai-conversation-id': 'realtime-session-123',
      'x-otel-user-id': 'user-456',
    },
  },
);

ws.addEventListener('open', () => {
  ws.send(
    JSON.stringify({
      type: 'session.update',
      session: { model: 'gpt-4o-realtime-preview' },
    }),
  );
});

ws.addEventListener('message', (event) => {
  const message = JSON.parse(event.data);
  console.log(message);
});
```

The model query parameter can be provider-qualified (`openai/gpt-4o-realtime-preview`) or a plain
model ID if a project Model Policy resolves it. Provon strips the provider prefix before forwarding
to OpenAI-compatible upstreams and adjusts the path for Azure targets.

## Client Secret

OpenAI's client SDKs often expect an ephemeral secret from `/realtime/client_secrets`. The Gateway
normalizes the request body, resolves the model, and forwards it to the upstream provider:

```bash
curl -X POST "$PROVON_API_URL/gateway/v1/realtime/client_secrets" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-realtime-preview",
    "session": {
      "model": "gpt-4o-realtime-preview",
      "modalities": ["audio", "text"]
    }
  }'
```

## Calls And Transcription Sessions

`/realtime/calls` and `/realtime/transcription_sessions` follow the same normalization rules as
`/realtime/client_secrets`. Send the OpenAI-compatible body; Provon resolves the nested model fields
and routes to the selected target.

## Fallback And Recovery

Realtime WebSocket connections support ordered fallback across multiple eligible Provider Keys. If
the first upstream WebSocket cannot be established, the Gateway tries the next prepared connection.
Once the bridge is active, recovery is no longer possible without reconnecting the client.

Configure fallback in the Provider Key's `fallbackPolicy` as described in
[Routing and reliability](./routing.md#reliability-configuration).

## Error Codes

| Code                                  | Status | Meaning                                                 |
| ------------------------------------- | ------ | ------------------------------------------------------- |
| `REALTIME_WEBSOCKET_REQUIRED`         | `426`  | The request did not include a WebSocket upgrade header  |
| `REALTIME_WEBSOCKET_UNAVAILABLE`      | `501`  | The runtime cannot terminate WebSocket upgrades         |
| `REALTIME_MODEL_REQUIRED`             | `400`  | The `model` query parameter is missing                  |
| `UPSTREAM_NOT_CONFIGURED`             | `404`  | No eligible Provider Key mapping for the realtime model |
| `REALTIME_UPSTREAM_CONNECTION_FAILED` | `502`  | The upstream WebSocket could not be established         |
| `GATEWAY_PAUSED`                      | `503`  | The project Gateway is paused                           |

## Limitations

- Realtime uses raw WebSocket passthrough. The Gateway does not inspect or rewrite individual
  realtime events after the bridge is established.
- Request/response guardrails that act on the full body run only on the setup handshake, not on
  each streaming event.
- Trace capture records the session lifecycle and upstream attempt, not per-event telemetry.
- Node deployments require a runtime or fronting layer that can upgrade HTTP to WebSocket.