Automations

Use an Automation when the response to an event is deterministic and repeatable. Use a diagnostic Rule when the primary job is to decide whether conversation evidence supports a product failure.

Choose The Right Path#

Automations do not replace the Rule, Run, Finding model. A workflow can create or consume a Finding, but Findings remain the canonical review and repair record.

Lifecycle#

An Automation has an editable definition and an immutable published version:

flowchart LR
  draft["Draft graph"] --> publish["Publish"]
  publish --> active["Active version"]
  active --> trigger["Trigger"]
  trigger --> queued["Queued Run"]
  queued --> steps["Step Runs"]
  steps --> result["Artifacts and side effects"]

The definition status is:

Publishing validates the graph against the operations available in the current runtime. A draft can be saved before every connector or runtime capability is configured; it cannot be published until the executable graph is valid.

Triggers#

Each Automation starts with one trigger:

Trace filters operate on normalized summary fields instead of scanning raw payload text. Prefer a small, explicit filter that you can verify against a representative trace.

Scheduled triggers use an IANA timezone. Configure whether a missed occurrence should fire once or be skipped when the scheduler resumes.

Operations#

The currently implemented operation families are:

Operations exchange typed artifacts. For example, an evaluator can emit a score and judgement, which can feed a Finding, notification, or dataset operation. Publishing rejects incompatible connections before a Run starts.

See Evals, Datasets, and Connectors for the domain-specific contracts.

Build A First Automation#

Start with a manual workflow before introducing an event trigger:

1. Open Automations in a project.
2. Create a workflow with a Manual trigger.
3. Add one side effect, such as Create a finding or Send Slack message.
4. Connect the trigger output to the operation input.
5. Publish the workflow.
6. Run it with a small representative input.
7. Open the Run and verify every Step Run and external result.

Once that path works, replace the manual trigger with a trace, schedule, or Gateway usage trigger. This isolates graph and connector errors from trigger-matching errors.

Runs And Reliability#

A Run records:

• the published workflow version;
• trigger type and trigger reference;
• normalized input and output;
• status, attempt count, and timing;
• each Step Run, including its input, output, and error.

Run statuses are queued, running, completed, failed, dead_letter, and cancelled.

Manual API Runs accept an Idempotency-Key. Repeating the same key for the same workflow addresses the same logical invocation instead of intentionally creating unrelated work. Automated triggers also derive stable trigger references so scheduler and event retries do not multiply side effects.

Retry a failed, dead_letter, or cancelled Run only after its dependency is healthy. Resume is for queued or failed work that should continue without creating a new logical invocation.

Connector Boundary#

Connectors hold credentials and destination defaults; workflow definitions hold references and field mappings.

• Do not place connector tokens, provider keys, or project API keys in workflow parameters.
• Keep external destination IDs explicit so a published workflow cannot silently change targets.
• Use dedupe fields for data destinations that support upsert.
• Treat external rows and notifications as projections, not the source of truth for Findings.

GitHub has two distinct roles:

• Finding repair handoff creates a repair Issue and tracks its resolution on the Finding.
• Automation issue creation is a general side effect and does not automatically become the Finding's repair reference.

Production Checklist#

• Verify one manual Run before enabling an automated trigger.
• Confirm the published version, not only the editable draft.
• Bound trace filters and schedule frequency.
• Configure connector credentials outside the workflow graph.
• Use idempotency or destination dedupe for side effects.
• Review failed and dead-letter Runs.
• Keep notification payloads free of secrets and unnecessary trace content.
• Pause the workflow before changing a high-impact destination.

API Access#

Service clients can manage definitions and Runs through the Workflow API.

Use a project API key with:

The default project API key includes workflows:read, not workflows:write.

Related Docs#

• Findings
• Evals
• Datasets
• Connectors
• Workflow API
• Troubleshooting