Knowledge Best Practices

Write focused Learning Goals#

Each goal should capture one kind of information. Split goals when the retention, review, or consumption needs differ.

A focused goal makes it easier to judge whether each extracted item is correct.

Structure the instruction#

A good instruction has three parts:

1. What to capture — the specific facts, constraints, or preferences.
2. What to ignore — temporary requests, contradictions, unsupported inferences, secrets.
3. How to phrase the result — concise title and content style.

Example:

Extract explicit user preferences about response format, length, or tone.
Ignore one-off requests that contradict earlier preferences.
Never infer preferences the user did not state.
Title should be under 10 words; content should state the preference and the supporting evidence.

Avoid unsupported inferences#

Provon extracts from the conversation summary and sampled events, not from domain knowledge or external context. If the instruction asks for inference beyond the evidence, items will be inconsistent and hard to trust.

Bad instruction:

Extract the user's favorite programming language.

Better instruction:

Extract explicit statements where the user names a preferred programming language for the project.

Use confidence as a review signal, not a probability#

The confidence field reflects how strongly the extraction model believes the candidate satisfies the goal, given the conversation. It is not a calibrated probability.

• High confidence (~0.8-1.0): Likely a direct statement. Still verify against the evidence.
• Medium confidence (~0.5-0.8): Interpretation is involved. Read the source conversation.
• Low confidence (<0.5): Often an inference or partial match. Either rephrase the goal or archive the item.

If many items have low confidence, make the instruction more specific or split the goal.

Manage goal lifecycle#

• active: participates in automatic extraction and can be backfilled.
• paused: excluded from automatic extraction. Useful when you are tuning the instruction and do not want noise from new traces. A paused goal can still be updated and reactivated.
• archived: permanently excluded from extraction. Existing items remain visible unless you archive them individually.

Pause a goal while iterating on its instruction, then backfill a small time window to validate the change before reactivating it.

Review every item before consumption#

Each item carries evidenceRefs that point back to the source conversation. Before treating an item as project truth:

1. Open the linked conversation or trace.
2. Confirm the title and content are supported by the evidence.
3. Archive items that are one-off, contradictory, or unsupported.
4. Edit items to make the statement more precise, not broader.

Keep items actionable for downstream consumers#

Knowledge is consumed by people, coding agents, and Dataset collection policies. Write item content that is useful without the original conversation:

• State the fact or constraint explicitly.
• Include enough context for a reader who was not in the conversation.
• Avoid pronouns or references that only make sense with the trace open.

Connect Knowledge to Datasets#

After reviewing items, use them to build training Examples with the teach_project_knowledge collection objective. This creates model-facing examples grounded in selected Learning Goals. See Datasets best practices for review and tagging guidance.

Prefer local search for agent context#

Use provon knowledge pull to materialize active Knowledge as Markdown, then use provon knowledge find to retrieve focused context for a coding agent:

provon knowledge pull
provon knowledge find "deployment region" --format paths

This keeps the agent context small and deterministic. It also lets you version the snapshot with your repository.

Common mistakes#

• Vague instructions such as "extract useful things" produce noisy, inconsistent items.
• Overly broad goals make review difficult because the category boundary is unclear.
• Ignoring low-confidence items lets unsupported facts enter the knowledge base.
• Forgetting to pause during iteration floods the inbox with items extracted from an old instruction version.

See also#

• Knowledge quickstart — first end-to-end extraction.
• Knowledge troubleshooting — debug missing or poor-quality items.
• Datasets best practices — turn reviewed Knowledge into training data.