Skip to content

Dataset Export Formats

Provon stores Examples in a consumption-neutral shape. When you want to train or evaluate, the Examples are converted to one of several standard formats. Each format validates that

View as Markdown Open the plain-text version of this page.

Format Quick Reference#

Format Input type Expected output Best for
openai_chat_sft chat chat_message SFT training with chat-formatted messages
chat_prompt_completion any any Legacy prompt/completion training pipelines
alpaca_instruction any any Instruction-tuning with { instruction, input, output }
preference_chat_dpo chat chat_message DPO, ORPO, KTO, and other preference methods
evaluation_jsonl any optional Running evals or inspecting Examples outside Provon

openai_chat_sft#

Produces a JSONL file where each line is a single chat-formatted conversation ending with the assistant message to learn.

Requirements#

  • payload.input.type must be chat.
  • payload.expectedOutput.type must be chat_message with role: assistant.

Example#

json
{
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "What is the capital of France?" },
    { "role": "assistant", "content": "The capital of France is Paris." }
  ]
}

Use With#

  • SFT training jobs (method: sft).
  • Any trainer that expects OpenAI-style chat completions training data.

chat_prompt_completion#

Produces a JSONL file with prompt and completion strings. The input and output are flattened to text, so this format works with non-chat schemas as well.

Requirements#

  • payload.expectedOutput is required.

Example#

json
{
  "prompt": "What is the capital of France?",
  "completion": "The capital of France is Paris."
}

Use With#

  • Older fine-tuning pipelines that expect prompt/completion pairs.
  • Quick experiments where exact chat structure is not required.

alpaca_instruction#

Produces a JSONL file with the Alpaca-style fields instruction, input, and output.

Requirements#

  • payload.expectedOutput is required.

Mapping Rules#

Provon input type instruction input
instruction instruction context or empty string
prompt prompt system or empty string
chat, text, json flattened text empty string

Example#

json
{
  "instruction": "Answer the user's geography question.",
  "input": "What is the capital of France?",
  "output": "The capital of France is Paris."
}

Use With#

  • Instruction-tuning datasets and trainers that expect the Alpaca schema.

preference_chat_dpo#

Produces a JSONL file with prompt, chosen, and rejected arrays of chat messages. This is the format used by Direct Preference Optimization (DPO) and related preference methods.

Requirements#

  • payload.input.type must be chat.
  • payload.expectedOutput.type must be chat_message with role: assistant.
  • payload.rejectedOutput.type must be chat_message with role: assistant.
  • The chosen and rejected messages must differ.

Example#

json
{
  "prompt": [{ "role": "user", "content": "What is the capital of France?" }],
  "chosen": [{ "role": "assistant", "content": "The capital of France is Paris." }],
  "rejected": [{ "role": "assistant", "content": "France is a country in Europe." }]
}

Use With#

  • DPO (method: dpo), ORPO (method: orpo), and KTO (method: kto) training jobs.
  • Any trainer that consumes prompt/chosen/rejected preference triples.

evaluation_jsonl#

Produces a JSONL file that preserves the full Example structure, including source, tags, and rubric. This format is designed for evaluation and auditing rather than training.

Requirements#

None. Examples are exported as-is. expectedOutput, rejectedOutput, and rubric are included when present.

Example#

json
{
  "id": "dsex_123",
  "input": {
    "type": "chat",
    "messages": [{ "role": "user", "content": "What is the capital of France?" }]
  },
  "expectedOutput": {
    "type": "chat_message",
    "message": { "role": "assistant", "content": "The capital of France is Paris." }
  },
  "source": {
    "kind": "conversation",
    "traceIds": ["trace_123"],
    "conversationId": "conversation_123"
  },
  "tags": ["objective:preserve_successful_behavior", "reviewed"]
}

Use With#

  • Offline evaluation scripts that need provenance and tags.
  • Human review workflows outside Provon.

Validation Errors#

If an Example does not satisfy a format's requirements, the export fails with a clear message. Common errors include:

  • input must be chat for openai_chat_sft or preference_chat_dpo.
  • expectedOutput must be an assistant chat_message when the output is text or missing.
  • rejectedOutput must be an assistant chat_message for preference_chat_dpo.
  • expectedOutput is required for chat_prompt_completion and alpaca_instruction.

Fix the Example in the Workbench or via the API, then retry the export or fine-tuning job.

Sampling And Run Manifests#

Fine-tuning does not export the mutable Dataset directly. It selects compatible Examples, applies deterministic stratified sampling, and freezes the result into an immutable run manifest. The manifest records:

  • the Dataset ID and schema;
  • the export format;
  • the selected Example IDs;
  • sampling metadata such as candidate count, selected count, and limit.

Because the manifest is immutable, later edits to the Dataset do not change the run.