# Integrate SDKs With The Gateway

Provon AI Gateway works with clients that can call an OpenAI-compatible API with a custom base URL
and Bearer token. Most migrations change only the endpoint and API key; model IDs and diagnostic
context remain explicit application choices.

Complete the [Gateway quickstart](./gateway-quickstart.md) first so the provider, project key, and
trace path are known to work.

## Choose An Integration

| Client shape                                   | Start here                                                         |
| ---------------------------------------------- | ------------------------------------------------------------------ |
| Any language or no SDK                         | [Direct HTTP](#direct-http)                                        |
| Existing OpenAI Python application             | [OpenAI Python SDK](#openai-python-sdk)                            |
| Existing OpenAI JavaScript application         | [OpenAI JavaScript SDK](#openai-javascript-sdk)                    |
| Anthropic-native Messages payloads             | [Native Messages](#native-messages-requests)                       |
| LangChain, LlamaIndex, Vercel AI SDK, or other | [Compatible frameworks](#frameworks-and-openai-compatible-clients) |

Provon does not require a Provon-specific inference SDK. The stable contract is the Gateway base
URL, Bearer authentication, request model, and optional evidence-context headers.

## Shared Configuration

```bash
export PROVON_GATEWAY_URL="http://127.0.0.1:3000/gateway/v1"
export PROVON_API_KEY="your_project_api_key"
```

`PROVON_API_KEY` is a Provon project API key with `gateway:invoke`. Do not put an upstream provider
key in the application.

Clients that read the standard OpenAI environment variables can often be redirected without code
changes:

```bash
export OPENAI_BASE_URL="$PROVON_GATEWAY_URL"
export OPENAI_API_KEY="$PROVON_API_KEY"
```

Use a provider-qualified model such as `openai/gpt-5-mini` during migration. Introduce plain model
names or `provon/auto` only after routing policy has been tested.

## Direct HTTP

Any HTTP client can call the Gateway:

```ts
const response = await fetch(`${process.env.PROVON_GATEWAY_URL}/chat/completions`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.PROVON_API_KEY}`,
    'Content-Type': 'application/json',
    'x-otel-gen-ai-conversation-id': 'conversation-123',
    'x-otel-gen-ai-agent-id': 'support-agent',
  },
  body: JSON.stringify({
    model: 'openai/gpt-5-mini',
    messages: [{ role: 'user', content: 'Summarize this ticket.' }],
  }),
});

if (!response.ok) throw new Error(await response.text());
console.log(await response.json());
```

Log the `x-provon-request-id` response header with the application request:

```ts
console.log(response.headers.get('x-provon-request-id'));
```

## OpenAI Python SDK

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PROVON_API_KEY"],
    base_url=os.environ["PROVON_GATEWAY_URL"],
    default_headers={
        "x-otel-gen-ai-agent-id": "support-agent",
        "x-otel-gen-ai-agent-name": "Support Agent",
    },
)

response = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=[{"role": "user", "content": "Summarize this ticket."}],
    extra_headers={
        "x-otel-gen-ai-conversation-id": "conversation-123",
        "x-otel-user-id": "user-456",
    },
)

print(response.choices[0].message.content)
```

The same client can use the Responses API when the selected target supports it:

```python
response = client.responses.create(
    model="openai/gpt-5-mini",
    input="Summarize this ticket.",
    extra_headers={
        "x-otel-gen-ai-conversation-id": "conversation-123",
        "x-otel-user-id": "user-456",
    },
)

print(response.output_text)
```

## OpenAI JavaScript SDK

```ts
import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.PROVON_API_KEY,
  baseURL: process.env.PROVON_GATEWAY_URL,
  defaultHeaders: {
    'x-otel-gen-ai-agent-id': 'support-agent',
    'x-otel-gen-ai-agent-name': 'Support Agent',
  },
});

const response = await client.chat.completions.create(
  {
    model: 'openai/gpt-5-mini',
    messages: [{ role: 'user', content: 'Summarize this ticket.' }],
  },
  {
    headers: {
      'x-otel-gen-ai-conversation-id': 'conversation-123',
      'x-otel-user-id': 'user-456',
    },
  },
);

console.log(response.choices[0]?.message.content);
```

Streaming uses the SDK's normal `stream` option. Provon preserves the provider-compatible stream
and records final usage and attempt evidence when the provider exposes it.

## Frameworks And OpenAI-Compatible Clients

For LangChain, LlamaIndex, Vercel AI SDK, or another framework:

1. Select its OpenAI or OpenAI-compatible provider adapter.
2. Set the base URL to `PROVON_GATEWAY_URL`.
3. Set the API key to `PROVON_API_KEY`.
4. Use a model ID available in the Provon capability matrix.
5. Add the context headers through the framework's default-header or per-request option.

Framework configuration names change across releases. The invariant contract is:

```text
Base URL:      https://<gateway-origin>/gateway/v1
Authorization: Bearer <Provon project API key>
Model:         <provider>/<model>, plain policy model, or provon/auto
```

Do not add a second `/v1` when the configured Gateway URL already ends in `/gateway/v1`.
The client must be able to send `Authorization: Bearer <Provon project API key>`. A client that
hard-codes provider-specific authentication without a header override is not drop-in compatible.

## Native Messages Requests

Provon also exposes `/gateway/v1/messages` for provider-native Messages payloads. Use Bearer
authentication even when the upstream provider normally uses `x-api-key`:

```bash
curl "$PROVON_GATEWAY_URL/messages" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -H "x-otel-gen-ai-conversation-id: conversation-123" \
  -d '{
    "model": "anthropic/claude-haiku-4.5",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Summarize this ticket."}]
  }'
```

The selected target must declare the `messages` endpoint and native passthrough capability. Query
the [capability matrix](./gateway-api.md#endpoint-discovery) instead of assuming parity.

## Diagnostic Context

Stable context makes individual model calls useful to conversation diagnostics:

| Purpose      | Header                          |
| ------------ | ------------------------------- |
| Conversation | `x-otel-gen-ai-conversation-id` |
| User         | `x-otel-user-id`                |
| Session      | `x-otel-session-id`             |
| Agent        | `x-otel-gen-ai-agent-id`        |
| Agent name   | `x-otel-gen-ai-agent-name`      |
| Agent build  | `x-otel-gen-ai-agent-version`   |
| Workflow     | `x-otel-gen-ai-workflow-name`   |

Use the same conversation ID for every turn in one user goal. Use non-secret, pseudonymous values;
these headers become retained trace attributes.

Set request-scoped values such as conversation, session, and user IDs on each request. Do not place
one user's identifiers in process-wide default headers. Agent name and version can be client-wide
when one client instance represents one deployed agent.

## Migration Checklist

1. Connect one Provider Key and keep **Request traces** set to **All**.
2. Verify the application endpoint ends in `/gateway/v1`.
3. Replace the provider key with a Provon project key.
4. Start with a provider-qualified model.
5. Send stable conversation and agent identifiers.
6. Test streaming, tools, structured output, and multimodal inputs separately.
7. Verify the root and provider-attempt spans in **Traces**.
8. Add limits, guardrails, and recovery policy before increasing traffic.

For rollout and rollback gates, continue with
[Run The Gateway In Production](./gateway-production.md).

## Related Docs

- [Gateway quickstart](./gateway-quickstart.md)
- [Migration guide](./gateway-migration.md)
- [Gateway API](./gateway-api.md)
- [Model providers](./model-providers.md)
- [Routing and reliability](./gateway-routing.md)
- [Gateway evidence](./gateway-observability.md)
- [Gateway troubleshooting](./gateway-troubleshooting.md)