Skip to content

Troubleshooting

Start with the first failed verification step. A Gateway response, an ingested trace, a diagnostic Run, a Finding, and a GitHub handoff are separate stages.

View as Markdown Open the plain-text version of this page.
On this page Browse sections

Provon Does Not Start#

Check:

bash
node --version
pnpm --version
pnpm dev
  • Node.js must be 22 or newer.
  • The local Workbench port defaults to 3000; the development API uses a separate internal port.
  • Another process or Provon instance must not hold the configured DuckDB file lock.
  • Relative local storage paths resolve from the repository root.

Use a custom Workbench port when needed:

bash
pnpm dev -- --port 3100

The Workbench Loads But API Calls Fail#

  • Read the pnpm dev output for the Workbench and API origins.
  • Confirm PROVON_API_ORIGIN was not overridden to an unavailable server.
  • When using a reverse proxy, forward the original host and scheme.
  • Confirm the signed-in user belongs to the requested project.

OTLP Returns 401 Or 403#

Verify the header format:

bash
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $PROVON_API_KEY"

The key must be a project API key with telemetry:ingest. Provider API keys and GitHub tokens are not valid OTLP credentials.

OTLP Succeeds But No Trace Appears#

Check these in order:

  1. The exporter uses OTLP/HTTP, not OTLP/gRPC.
  2. The endpoint is the Provon origin, or the signal-specific URL ends in /v1/traces.
  3. The exporter is flushed before the process exits.
  4. The trace is in the same project as the API key.
  5. The selected Workbench time range includes the span timestamps.

Cloudflare ingestion is asynchronous. A 200 means the payload was durably staged and queued; trace summaries can appear after the raw write completes.

The background worker parses and validates individual records after the response. If a known trace never appears, inspect ingest-worker and dead-letter logs for malformed OTLP envelopes or rejected records.

OTLP Returns 413, 422, 501, Or 503#

  • 413: reduce the exporter batch size. The default decoded request limit is 10 MiB.
  • 422: use a supported content type and content encoding, and confirm protobuf decoding is configured.
  • 501: the deployed runtime does not enable the requested signal.
  • 503: retry with exponential backoff and honor Retry-After when queue admission is saturated.

Do not retry authentication or validation errors without changing the request.

See the OTLP/HTTP API for the complete acceptance and retry contract.

Traces Appear Without Conversation Grouping#

Add a stable gen_ai.conversation.id to the agent spans. Reusing a trace ID is not a substitute for conversation identity because one conversation may contain multiple traces. session.id is retained as an attribute but is not the canonical Provon conversation key.

Also verify:

  • the identifier is attached consistently across turns;
  • timestamps and parent span IDs preserve event order;
  • user, assistant, and tool content was retained when diagnosis requires it.

Provon also recognizes conversation.id, conversation_id, provon.conversation_id, thread.id, and thread_id.

A Trace Is Visible But Not Diagnosable#

Trace visibility only requires valid spans. Diagnostic Rules need a reconstructable trajectory. Verify that the trace includes:

  • a bounded root operation and ordered child spans;
  • the user goal in gen_ai.input.messages;
  • the terminal answer in gen_ai.output.messages;
  • tool call IDs, arguments, results, failures, and retries;
  • error span status plus error.type;
  • raw evidence allowed by the payload-retention policy.

See Trace model and instrumentation. Use the diagnosis-ready tracing checklist to review the complete run.

Trace Search Misses An Older Trace#

Trace list and search default to the last 24 hours. Select a wider Workbench range or pass a larger API or CLI range such as --range 7d.

Content, input, and output search uses projected previews and remains available with projected-only retention. It can match only the retained previews; the full raw body is unavailable for later inspection.

Agent Transcript Sync Finds No Files#

  • Point provon sync at a directory containing .json or .jsonl transcripts.
  • Use --once --dry-run --print-sample to inspect detection without uploading.
  • Confirm the process can read nested source directories.
  • Wait for an actively written JSONL record to finish; the scanner ignores an incomplete final line until the file has been quiet long enough.
  • Inspect the command's failures and warnings arrays.

AI Gateway#

Use Gateway troubleshooting for the request-ordered diagnostic path:

  1. verify project API-key authentication and capability;
  2. inspect model resolution and project policy;
  3. inspect the selected provider attempts;
  4. verify Trace Capture and telemetry delivery.

It covers 401, 402, 403, 404, 409, 413, 415, 429, and 503 responses, streaming failures, provider errors, missing traces, and incomplete conversation evidence.

Auto-Diagnose Produces No Findings#

This can be correct. Healthy conversations and weak candidates should not create Findings.

Verify:

  1. Auto-diagnose is enabled. New projects default to disabled.
  2. The relevant Rules are enabled.
  3. The conversation has been inactive for at least 10 minutes.
  4. A stable conversation identifier and usable trajectory evidence exist.
  5. Rule keywords match the captured conversation text.
  6. Signal and diagnostic-confidence thresholds are not filtering the candidate.
  7. Model adjudication has a usable model when the Rule mode is auto.

To separate scheduling from detection, open Rules and backfill the last 24 hours. Use deterministic-only when testing without an adjudication model.

A Rule Run Is Skipped#

Common skip reasons:

  • the Rule was disabled;
  • automatic scheduling was turned off;
  • the conversation no longer exists;
  • newer activity superseded the queued snapshot;
  • no usable trajectory could be reconstructed;
  • configured keywords did not match.

Skipped Runs do not publish Findings.

Create Repair Issue Fails#

The current repair handoff requires an enabled GitHub connector.

Check:

  • Connectors > GitHub is enabled;
  • a token or OAuth connection is stored;
  • Default owner and Default repo are set;
  • the token can create issues in that repository;
  • the deployment has an integration secret for encrypted connector credentials.

For OAuth, configure GITHUB_INTEGRATION_CLIENT_ID and GITHUB_INTEGRATION_CLIENT_SECRET. For local testing, expand the connector and provide a token manually.

A Finding Is Handed Off But Never Resolves#

Provon stores references as github:owner/repo#number. Resolution sync needs:

  • the GitHub connector to remain enabled;
  • permission to read the issue and associated pull requests;
  • the background repair-status loop to be running;
  • either a merged associated pull request or a closed issue.

Closing a Finding in the Workbench is not a substitute for the external repair outcome.

Seed Data Fails#

pnpm seed:local intentionally requires empty metadata and telemetry databases. It will not merge demo data into an existing workspace.

Stop the local server before seeding so DuckDB is not locked. Use a separate configured database if you need to preserve existing local data.

Still Blocked#

Capture:

  • the failing command or HTTP method and path;
  • the response status, error code, and request ID;
  • the runtime shape (Node or Cloudflare);
  • the relevant project ID;
  • whether the previous stage in the loop succeeded.

Do not include project API keys, provider credentials, GitHub tokens, prompts, or private payloads in a public issue.