Overview
The Core Concepts pages define the entities. These guides explain how the pieces work together, then let you try each stage. Examples show the JavaScript SDK first and the equivalent CLI path second wherever both are supported.
The loop at a glance
| Stage | What you do | Primary concepts |
|---|---|---|
| Connect | Package your agent as a recipe, register a runtime, point staging at it, run a first task. | Recipes, Runtimes, Environments, Tasks |
| Observe | Every task becomes a conversation; the platform synthesizes observations on completion and clusters them into patterns. | Conversations, Observations & Patterns |
| Investigate | Drill top-down (patterns → observations → conversations) with structured filtering. | Observations & Patterns, Conversations |
| Evaluate | Write a judge: a git-authored YAML rubric that grades each conversation pass / fail / not_applicable at teardown, and trend it across versions. | Judges |
| Experiment | When before/after is confounded, run a live A/B between runtime versions, scored by a judge’s pass-rate. | Experiments |
| Ship | Choose from the evidence and merge that recipe’s PR; the main push activates production. | Recipes, Environments |
| Guard | Leave the judge running as a standing regression guard so quality can’t silently drift. | Judges |
Why a loop, not a checklist
The stages feed each other. Observations surface a failure mode; a judge encodes the standard the agent should have met; an experiment tests the fix without a confounded before/after; and shipping moves the measured version into production. The same judge stays on as a guard, so the next regression appears as a changed number and starts the loop again from a known baseline.
What stays in git vs. what’s live
A recurring theme across these workflows: behavior lives in git, operation lives on the platform.
- In git: the recipe (instructions, profiles, skills) and judge rubrics. Iterating means a new commit, which is a new immutable recipe.
- On the platform: runtimes, environment assignments, experiments, and a judge’s operational state (its off-switch and production sample rate).
This split makes every measurement traceable: a quality number always names the exact recipe commit that produced it, and changing how you run a judge never disturbs what it measures.
Run the four guides in order
- Connecting an agent: validate → runtime →
staging→ first task. - Observing & investigating: conversation → pattern → observation → full transcript.
- Evaluating with judges: fixture → local evaluation → deployed judgement → judge ID.
- Experimenting & shipping: runtime versions → experiment → evidence → merge → production verification.
In CLI tabs, jq only moves fields from one JSON response into the next command. The CLI’s own --query flag is useful for inspection, but its JSON scalar output is quoted and is not a shell-safe raw value.
Project setup and GitHub integration happen outside these guides, and production changes when a recipe PR is merged. The CLI covers the operational steps between those Git boundaries, but it does not create integrations or merge pull requests.
Next
- Connecting an agent: start the loop.
Related
- Introduction: the operator loop, framed.
- Core Concepts: the entities these workflows act on.
- JavaScript SDK: the primary programmatic surface used by the first tab.
- CLI: command reference, output contract, and unsupported boundaries.