Knowledge

When To Use It#

Use knowledge extraction when you want to:

• capture reusable facts, constraints, or preferences observed in agent behavior;
• build a project knowledge base from production traces without manual curation;
• feed extracted items into downstream workflows such as prompt improvement or fine-tuning Datasets.

Knowledge is intentionally category-agnostic. The meaning of an item is determined by the goal's instruction rather than a fixed taxonomy.

Concepts#

A goal can be active, paused, or archived. Only active goals participate in automatic extraction and can be backfilled.

Enable Extraction#

Open Knowledge in the Workbench and enable automatic extraction. New conversations become candidates after they are inactive.

Extraction applies every active Learning Goal independently. Start with one focused goal so the resulting items are easy to judge before adding broader instructions.

Manage Goals#

Create a Learning Goal with:

• a specific name;
• an instruction that states what to retain and what to ignore;
• an active, paused, or archived status.

For example:

Name: Output format preferences
Instruction: Extract explicit user preferences about response format, length, or tone.

Use separate goals for facts with different review or retention needs. Paused and archived goals do not participate in automatic extraction.

Backfill Existing Traces#

Select Scan past traces for an active goal and choose the evidence window. Backfills run asynchronously and use the same extraction behavior as new conversations.

Use an idempotency key when automating backfills through the API.

Review Items#

Each item includes the extracted content, confidence score, and evidenceRefs pointing back to the source trace and spans. Review the title and content against that evidence before treating it as project truth. Correct the item when the evidence supports a more precise statement, or archive it when it should no longer be consumed.

Use Knowledge#

Knowledge can guide prompt and workflow changes, provide local context to coding agents, and supply examples for a Dataset configured to teach project knowledge.

Materialize active items as managed Markdown:

provon knowledge pull
provon knowledge find "deployment region" --format paths

See Knowledge CLI for local consumption and the Knowledge API for automation.

Extraction Behavior#

• Extraction runs asynchronously after a conversation is inactive, similar to diagnostic Rules.
• A goal with no instruction or an inactive status produces no items.
• Items from repeated or similar evidence update the existing item rather than creating duplicates.
• Extraction requires the conversation trajectory to be present; spans alone are not always enough.

Limitations#

• Automatic extraction is only available when the runtime provides a KnowledgeExtractionService.
• Extraction quality depends on the goal instruction; ambiguous instructions produce inconsistent items.
• Knowledge is project-scoped; there is no automatic sharing across projects or organizations.

Next Steps#

• Datasets for collecting examples that teach reviewed project knowledge.
• Findings for the diagnostic workflow that complements Knowledge.
• Tracing for improving the evidence available to extraction.
• Knowledge API for settings, goals, items, and backfills.