# Migrate To The Provon Gateway

This guide moves an existing model call behind Provon without changing model behavior and routing
policy at the same time. The migration is complete only when the application response and Provon
trace both pass verification.

Complete the [Gateway quickstart](./gateway-quickstart.md) against the target project first.

## Migration Strategy

Change one responsibility at a time:

1. Keep the current provider and model.
2. Move the provider credential into Provon.
3. Change the application base URL and API key.
4. Verify response compatibility and Gateway trace evidence.
5. Add stable conversation and agent context.
6. Add logical model names, recovery, Auto Router, limits, or guardrails only when needed.

Use a provider-qualified model during steps 1 through 5. For example, migrate `gpt-5-mini` to
`openai/gpt-5-mini`, not to `provon/auto`.

## 1. Inventory The Current Calls

Record every request shape the application uses:

| Field                    | Examples                                                  |
| ------------------------ | --------------------------------------------------------- |
| Endpoint                 | Chat Completions, Responses, Messages, embeddings, rerank |
| Provider and model       | OpenAI and `gpt-5-mini`                                   |
| Features                 | Streaming, tools, structured output, vision, audio        |
| Authentication           | Bearer, `x-api-key`, cloud-specific signing               |
| Client behavior          | Timeout, cancellation, retries, error parsing             |
| Payload limits           | Largest request, response, file, image, or audio body     |
| Required provider fields | Provider-specific body fields or response metadata        |

Do not assume that an OpenAI-compatible chat request implies compatibility for files, batches,
realtime, fine-tuning, or provider-native extensions.

## 2. Verify Target Capability

Connect the upstream credential under **Providers**, then query the exact endpoint and feature set:

```bash
curl \
  "$PROVON_BASE_URL/v1/gateway/model-selection?endpoint=chat-completions&features=streaming,tool-calling" \
  -H "Authorization: Bearer $PROVON_API_KEY"
```

This discovery call requires `workspace:read`. Inference requires `gateway:invoke`.

Check every request shape independently. A provider may be native for one endpoint, translated for
another, bridged through a different endpoint, or unsupported. See the
[Gateway API discovery contract](./gateway-api.md#endpoint-discovery).

## 3. Separate The Credentials

Before migration:

```text
application -> upstream provider key -> provider
```

After migration:

```text
application -> Provon project API key -> Gateway -> project Provider Key -> provider
```

Create a project API key for the application and environment boundary. Grant `gateway:invoke` and
only the other capabilities the workload needs. Store the upstream provider key in a
project-scoped Provider Key.

Rotate the two credentials independently. Never send the upstream key to the Gateway inference
endpoint.

## 4. Change The Client

For an OpenAI-compatible client, replace:

| Setting  | Before                      | After                                    |
| -------- | --------------------------- | ---------------------------------------- |
| Base URL | Provider API origin         | `<PROVON_ORIGIN>/gateway/v1`             |
| API key  | Upstream provider key       | Provon project API key                   |
| Model    | Provider model ID           | Provider-qualified Gateway model ID      |
| Auth     | Provider-specific mechanism | `Authorization: Bearer <PROVON_API_KEY>` |

Environment-only migrations commonly become:

```bash
export OPENAI_BASE_URL="$PROVON_BASE_URL/gateway/v1"
export OPENAI_API_KEY="$PROVON_API_KEY"
```

Do not append another `/v1` when the configured base URL already ends in `/gateway/v1`.

Clients that hard-code provider authentication, sign requests, or reject custom base URLs need an
adapter or direct HTTP integration. See [SDK integrations](./gateway-integrations.md).

## 5. Preserve Model Intent

Start with a provider-qualified model:

```json
{
  "model": "openai/gpt-5-mini"
}
```

This keeps provider intent explicit while Provon selects among eligible Provider Keys for that
provider.

After the migration is stable, choose deliberately:

| Desired behavior                            | Model form and configuration            |
| ------------------------------------------- | --------------------------------------- |
| Keep provider selection in application code | `<provider>/<model>`                    |
| Give the application a stable logical name  | Plain model plus a project Model Policy |
| Let Provon select across mapped targets     | `provon/auto` plus Auto Router          |

Changing to a logical model or Auto Router is a routing change, not part of the transport
migration. Test it separately.

## 6. Add Diagnostic Identity

Gateway capture can diagnose a single call without extra headers, but conversation Rules need
stable identity across turns.

Send request-scoped values:

```text
x-otel-gen-ai-conversation-id: one stable ID for the user goal
x-otel-user-id: a pseudonymous user ID
x-otel-session-id: the application session
x-otel-gen-ai-agent-id: the logical agent participant
x-otel-gen-ai-agent-version: the deployed agent version
x-otel-gen-ai-workflow-name: the workflow or graph name
```

Use the same conversation ID for every turn in one goal. Do not reuse one conversation ID across
unrelated users or tasks. Keep user, session, and conversation values out of process-wide default
headers when a client instance serves multiple requests.

See [Gateway evidence](./gateway-observability.md#add-diagnostic-context) and
[multi-agent tracing](./tracing-multi-agent.md#gateway-context).

## 7. Avoid Duplicate Model Evidence

If the application already emits an OpenTelemetry model-call span, decide which source owns that
operation:

- use the Gateway span as the model-boundary record; or
- keep the application model span and disable equivalent Gateway persistence.

Keep application-side spans for tools, handoffs, queues, retrieval, and business operations. The
Gateway cannot reconstruct work that never crosses the model boundary.

Recording the same model call twice can distort token, cost, latency, and diagnostic aggregates.
See [diagnosis-ready tracing](./tracing-best-practices.md).

## 8. Canary And Verify

Make the Gateway base URL a runtime configuration value and move a small cohort first.

For each canary request:

1. Compare status, response shape, output, and provider model with the direct path.
2. Test streaming, tools, structured output, and multimodal requests separately.
3. Log `x-provon-request-id` with the application request.
4. Open the trace and verify the root span and every provider attempt.
5. Check latency, time to first chunk, tokens, cost, and normalized errors.
6. Confirm conversation and agent identity are correct.

Do not enable fallback to hide a compatibility failure. First make the intended primary path pass.

## 9. Add Controls In Order

After compatibility is established:

1. Attach request, token, and cost limits to the application key.
2. Run new guardrails in **Monitor** mode.
3. Add bounded retries for transient failures.
4. Add same-model recovery before cross-model substitution.
5. Introduce Model Policies or Auto Router only for an explicit routing requirement.
6. Choose the steady-state Trace Capture and retention policy.

Each control changes behavior. Verify its client contract and trace evidence before adding the
next.

## Rollback

Keep these independent switches during the rollout:

- the prior provider base URL;
- the prior application secret reference;
- the provider-qualified model;
- a Gateway pause control;
- unchanged Provider Keys and mappings until the rollout is complete.

Rollback restores the previous base URL and provider credential. It does not delete Gateway traces,
so failed canaries remain available for diagnosis.

## Migration Gate

- Every endpoint and feature appears in discovery.
- Project and provider credentials are separate.
- A provider-qualified request passes before policy-based routing is enabled.
- Client timeouts, cancellations, streaming, and error parsing behave as expected.
- `x-provon-request-id` is logged.
- Gateway traces contain provider attempts and required diagnostic identity.
- Duplicate model-call telemetry has a defined owner.
- Limits and guardrails have been tested with expected rejections.
- The previous direct-provider path has been exercised as rollback.

Continue with the [production guide](./gateway-production.md). Use
[Gateway troubleshooting](./gateway-troubleshooting.md) when a verification gate fails.