Core Concepts

Everything in Introspection is organized around a few entities. This page defines each one and shows how they connect.

At the center is a simple idea: your agent’s behavior lives in git (a recipe), runs on Introspection (a runtime), and produces a record you can study and improve (conversations, observations, judgements, experiments). The sections below follow the operator loop: build, run, understand, improve.

Recipes

A recipe is your agent’s behavior, packaged and versioned in git. It is a precise pointer: a repository, a git ref, a commit SHA, and an optional sub-path. That package contains the agent’s instructions, profiles, skills, and judges.

Recipes are immutable: iterating on your agent means a new commit, which is a new recipe. Because every recipe is pinned to an exact commit, every run is reproducible, and you can compare any two versions with confidence.

Runtimes

A runtime is the deployable version of your agent: it binds a recipe to Introspection’s execution harness, along with the policy and resources it needs to run. A runtime serves one or more environment lanes (typically staging and production) so you can promote a version from test to live traffic.

Successive versions of the same logical agent belong to a runtime group, which gives each agent a stable lineage. That lineage is what lets experiments route between versions and lets patterns accumulate across them.

Environments

An environment is a lane (development, staging, or production) that cuts across the platform. Runtimes serve specific environments, API keys are scoped to one, and experiments run within a single lane. This keeps test traffic and production traffic cleanly separated.

Tasks

A task is one execution of a runtime: a single agent conversation. Tasks can be interactive (a live session you send follow-up messages to) or one-shot. Each task moves through a lifecycle (pending, queued, running, idle, completed, failed, cancelled; see Tasks) and runs in its own isolated sandbox.

Within a task, a task run is a segment of work opened against a runtime (or an experiment). Sending a new prompt, steering an in-progress run, or clearing context each open a run, so a single task can span several turns.

Conversations

A conversation is the captured, replayable record of a task: the full sequence of model calls, tool calls, and messages, reconstructed from the telemetry the runtime emits. Conversations are immutable and are the unit you investigate, judge, and experiment over.

You can replay any conversation turn by turn, and, where sharing is enabled, fork a new task from a shared conversation to reproduce and debug it.

Events, observations & patterns

These three layers turn raw agent traffic into something you can reason about.

An event is a single durable telemetry record. Events are the unified research surface: raw activity plus the platform-generated records below.
An observation is something the platform noticed about one completed conversation, along one of several lenses: user intent, task resolution, user sentiment, agent struggle, and environment issues.
A pattern is a named cluster of similar observations across many conversations. Patterns make recurring behavior and failure modes visible: instead of reading transcripts one by one, you start from “this happened 240 times this week” and drill in.

The investigation path runs top-down: patterns → observations → conversations.

Investigation gives you reproducible, filterable answers about agent behavior: the same query returns the same structured result every time.

Judges

A judge is how you encode what “good” looks like, as code. A judge is a version-pinned, LLM-as-judge rubric authored in your recipe’s git repository; it grades a conversation pass, fail, or not_applicable. Judges run automatically when a conversation completes.

Each verdict is a judgement, an immutable record you can trend over time. Because judges are pinned to a recipe version, you can see exactly how a quality metric moves as your agent changes, and keep a judge running as a standing regression guard.

Experiments

An experiment is a live A/B test between runtime versions in a single environment. You define arms (for example, a candidate recipe versus the current baseline), and Introspection routes traffic between them with sticky per-subject assignment, so a given user stays on one arm. The experiment is scored by a goal (typically a judge’s pass-rate) so you can ship the winner with evidence rather than a hunch.

Experiments have a simple lifecycle: run → start → end or cancel. Ending stops evidence collection; selecting and shipping a version is a separate Git decision.

Files, artifacts & shares

Runtimes read and write files. Anything a run writes to its outputs persists as a durable artifact you can download or feed into later work. A share is a read-only link to a file or conversation, and is what enables forking a task from a shared conversation.

Bindings

Bindings supply a runtime with what it needs at execution time, scoped by project, runtime group, runtime, or environment:

Endpoints: upstream HTTP/API targets (for example a model gateway), with encrypted headers.
Variables: sandbox environment variables and credentials.

Organizations, projects & members

An organization is the tenant root; everything is scoped to it.
A member is a user within an organization, with a role.
A project is the workspace most resources hang off: recipes, runtimes, experiments, tasks, conversations, and the rest.
An API key is the long-lived, environment-scoped credential server-side SDKs use. It carries the runtimes capability; the CLI authenticates separately as a member through browser-approved login.

Integrations (such as GitHub) and deployments are also configured at the organization level.

How it fits together


Organization
└── Project
    ├── Repository
    │   └── Recipe            (repo + ref + commit + sub-path: the agent's behavior)
    ├── Runtime group         (lineage of one logical agent)
    │   └── Runtime           (a deployable version; serves staging / production)
    │       └── Judges        (git-authored quality rubrics)
    ├── Experiment            (A/B between runtime versions)
    └── Tasks
        └── Conversation      (the replayable record)
            ├── Observations  → Patterns
            ├── Judgements
            └── Feedback