CLI

The CLI authenticates as a member through browser-approved device login. It is deliberately broader than an application SDK: SDKs read and run runtimes, while the CLI owns deployment and operator workflows such as bootstrapping a runtime, routing staging, deactivating a version, and configuring secrets.

For a linear, copyable runbook, start with Connecting an agent and continue through the four workflow guides.

Install

# npm installs the platform-specific binary.
npm install -g @introspection-ai/cli
 
# Or run it without a global install.
npx @introspection-ai/cli login

The npm package selects the matching platform binary through an optional dependency; there is no build step or postinstall download.

Log in

introspection login
introspection whoami

Login uses the OAuth 2.0 Device Authorization flow. The CLI stores the resulting member-bound session at ~/.introspection/credentials.json with mode 0600 and refreshes it transparently. whoami validates the session and prints the Control Plane/Data Plane targets, granted scope, organization, project, member, and expiry without printing tokens.

The server always chooses the CLI capabilities: runtimes for platform operations and remotes for introspection dev. There is no capability flag. Project API keys and applications cannot request the CLI-only remotes scope.

introspection logout

Command map

Run introspection <group> --help for the version-installed flag contract.

api-keys manages project API keys for application/SDK callers. applications manages application identities and their public keys. Neither can request the member-only remotes capability used by dev.

API keys and applications

Project API keys are environment-bound credentials for SDK callers. create writes the one-time secret to a new mode-0600 file, acknowledges it only after the write succeeds, and never prints the secret:

introspection api-keys create \
  --name staging-sdk \
  --environment staging \
  --secret-file ./staging-api-key
introspection api-keys list
introspection api-keys update --key <key-id> --name staging-sdk-2
introspection api-keys revoke --key <key-id>

If the file write succeeded but acknowledgement failed, retry api-keys acknowledge --key <key-id>. Use revoke --expires-in <seconds> for a grace period.

Applications are organization-managed callers. Create the type that matches the trust model, retain its id, then manage its subordinate resources:

introspection applications create --type spa --name web \
  --redirect-uri https://app.example.com/callback \
  --allowed-origin https://app.example.com
 
introspection applications create --type jwks --name backend \
  --issuer https://issuer.example.com
 
introspection applications create --type service-account --name worker \
  --allowed-scope runtimes
 
introspection applications list
introspection applications idps list --app <application-id>
introspection applications idps delete --app <application-id> --idp <idp-id>

For a service account, applications secrets create --app <application-id> returns its secret once. Protect stdout before persisting it; later secrets list never returns the value, and secrets revoke --app <id> --secret <id> invalidates it. Creating applications and federations can require an owner/admin member login.

Local development

Run from the Git working tree that contains your .introspection/<runtime-slug>.yaml manifest:

introspection dev

dev resolves the runtime from the Git repository (or --runtime), verifies that its cloud recipe is valid, checks declared MCP bindings, opens Runtime Preview, and serves local recipe changes to the development runtime. SDKs on the same machine can use the short-lived development session instead of a static API key.

import { IntrospectionClient } from "@introspection-sdk/introspection-node";
 
const client = new IntrospectionClient();
const runner = await client.runtimes("support-agent").run();

Missing required and optional development bindings warn by default. Use --check-bindings to turn missing required bindings into a failing readiness check and exit without attaching:

introspection dev --check-bindings
introspection dev --runtime support-agent

The CLI never reads local MCP credential files, uploads local secrets, or silently creates endpoints. Configure those through cloud bindings.

Validate recipes

introspection recipes validate checks the manifest, package metadata, agent YAML, and every declared agent, skill, extension, include, and glob boundary locally and offline.

# Discover every manifest under .introspection in the current workspace.
introspection recipes validate
 
# Validate a different workspace and show compact diagnostics.
introspection recipes validate --work-dir ../support-agent -o table

Use repeatable --path flags to validate specific manifests. JSON is the default output; error diagnostics exit 1, while warnings alone do not fail the command.

Cloud recipe records are immutable Git pins:

introspection recipes list [--name <name>]
introspection recipes create --name <name> \
  --repository-id <id> --git-ref <ref> --git-sha <commit> [--sub-path <path>]

Runtime bootstrap normally creates the initial recipe pin for you. Use recipes create only when operating the lower-level recipe record directly.

Bindings

introspection bindings is a CLI umbrella over three flat Control Plane resources. It is not a separate /v1/bindings API:

All three resource types can be project-wide, runtime-group scoped (--runtime-group), or exact-runtime scoped (--runtime). Add --environment development|staging|production for one lane; omit it for a shared row.

Endpoint and credential example

Endpoint create/update accepts a complete JSON or YAML document through --file; flags override document fields:

name: linear
url: https://mcp.example.com/mcp
kind: mcp
runtime_group_id: 019...
environment: production
metadata:
  mcp_server_id: linear
headers:
  Authorization: "Bearer ${LINEAR_TOKEN}"

Create the referenced credential without placing its value in shell arguments:

export LINEAR_TOKEN='...'
 
introspection bindings credentials create \
  --name LINEAR_TOKEN \
  --value-env LINEAR_TOKEN \
  --environment production
 
introspection bindings endpoints create --file linear-endpoint.yaml
 
export API_REGION='us-west-2'
introspection bindings variables create \
  --name API_REGION \
  --value-env API_REGION \
  --runtime-group 019... \
  --environment staging

Credential and variable commands can also read from a mode-0600 file or stdin. Credential reads expose has_secret and secret_set_at, never the value. Endpoint reads return the unresolved ${NAME} template and referenced credential names, never an expanded secret.

MCP connections

bindings mcp connect validates a stable recipe MCP id, then creates or upserts the concrete endpoint. list composes recipe declarations with the endpoints applicable to one runtime group and environment, including required-but-missing and stale connections.

introspection bindings mcp connect \
  --recipe <recipe-id> \
  --mcp-server-id linear \
  --name Linear \
  --endpoint-url https://mcp.example.com/mcp
 
introspection bindings mcp list \
  --recipe <recipe-id> \
  --runtime-group <runtime-group-id> \
  --environment production

Runtimes

Runtime lifecycle is manifest-driven and operator-owned:

introspection runtimes create --manifest .introspection/<name>.yaml
introspection runtimes list [--runtime <slug-or-group-id>]
introspection runtimes get <runtime-id>
introspection runtimes versions <runtime-id>
introspection runtimes staging track <runtime-id> --git-ref <main|pr/N>
introspection runtimes staging pin <runtime-id>
introspection runtimes deactivate <runtime-id> [--reason <text>]
introspection runtimes reactivate <runtime-id>

create reads the canonical manifest and Git checkout, creates the immutable recipe pin through the active GitHub integration, and bootstraps the first runtime. The manifest filename is the runtime-group slug. Later versions remain Git-driven.

staging track follows new commits on a branch or PR ref. staging pin freezes staging on the selected version. Deactivate/reactivate controls whether a runtime is available for new work without deleting its history.

Tasks

The CLI can create and operate tasks without writing an SDK program:

introspection tasks create --runtime <slug-or-group-id> --environment production [--prompt <text>]
introspection tasks create --runtime-id <runtime-id> --environment staging [--prompt <text>]
introspection tasks create --experiment <experiment-id> --subject <stable-id> [--prompt <text>]
introspection tasks prompt <task-id> --prompt <text>
introspection tasks follow <task-id> --run <run-id|current> --since <sequence>
introspection tasks resume <task-id> --interrupt-id <id> --payload '<json>' …
introspection tasks abort <task-id>
introspection tasks list [--status <status>] [--runtime <runtime-id>] [--next <cursor>]
introspection tasks get <task-id>
introspection tasks delete <task-id>

Task creation first mints a transient runner credential. The credential is used only to create and initially run the task and is never persisted.

follow writes one JSON object per AG-UI event and requires an explicit replay cursor (--since 0 for the first attach). If the requested sequence fell outside the bounded replay window, the CLI preserves the resume_gap event so the operator can detect the gap.

resume verifies a complete response batch against durable metadata.pending_interrupts; use repeatable --cancel-interrupt <id> entries for cancelled answers. abort cancels only the active run and then prints the task’s resulting state.

The minimal task lifecycle is:

introspection tasks create \
  --runtime support-agent \
  --environment staging \
  --subject smoke-1 \
  --prompt "Reply with ready." > task.json
 
TASK_ID=$(jq -r '.task.id' task.json)
RUN_ID=$(jq -r '.run.id' task.json)
introspection tasks follow "$TASK_ID" --run "$RUN_ID" --since 0 > events.jsonl
introspection tasks get "$TASK_ID" > task-final.json
jq -e '.status == "idle" or .status == "completed"' task-final.json

follow is always JSONL, regardless of -o. It ends when the run settles. Keep the highest received numeric sequence for reconnection; --since 0 is only the first attach. If task-final.json contains metadata.pending_interrupts, answer every pending ID in one resume call and follow the returned current run.

Experiments

The CLI exposes the full experiment lifecycle:

introspection experiments list [--runtime-group <id>] [--environment <env>] [--status <status>]
introspection experiments get <id>
introspection experiments create --file experiment.yaml
introspection experiments start <id>
introspection experiments end <id>
introspection experiments cancel <id>

create --file accepts the exact ExperimentCreate JSON or YAML document and injects the active project when omitted. The server remains the schema authority; the CLI performs only cheap structural checks before sending it.

A minimal valid document is:

name: support-agent-candidate
runtime_group_id: 019...
environment: production
sample_rate: 0.10
arms:
  - runtime_id: 019...
    arm_label: baseline
  - runtime_id: 019...
    arm_label: candidate
goal_json:
  kind: composite
  direction: maximize
  components:
    - source: judge
      judge_id: 019...
      judge_definition_hash: sha256:...
      weight: 1.0

All arms must belong to the named runtime group. Arms use runtime IDs and are symmetric: there is no create-time control arm or authored arm weight. end takes no winner; it stops evidence collection and does not deploy anything.

Judge calibration

Build labelled local fixtures from real conversations, then evaluate a judge through an OpenAI-compatible endpoint using the Rust judge engine:

introspection conversations get \
  --ids-file conversation-ids.txt \
  --output-file judge-fixtures.jsonl
 
OPENAI_API_KEY=... introspection judges eval \
  --judge judges/useful-support-resolution.yaml \
  --dataset judge-fixtures.jsonl

The fixture file owns expected (pass or fail) and optional split (train, dev, or test) labels. The judge YAML owns the model configuration:

judge: useful_support_resolution
description: Did the response move the support request toward a useful resolution?
llm:
  provider: openai
  model: gpt-4.1-nano
instructions: |
  Return pass when the response gives a grounded answer, asks for information
  needed to proceed, or clearly escalates with a next step. Return fail when it
  only repeats policy or leaves the customer stuck. Return not_applicable when
  the conversation is not a support request. Output { reasoning, verdict }.

OpenAI defaults to OPENAI_API_KEY; OpenRouter defaults to OPENROUTER_API_KEY. A custom local endpoint is explicit under llm.local. The Introspection login credential is never used for the model call, and model credentials are not written into fixtures or reports.

Telemetry reads

The CLI reads conversations and exactly one typed event family per request, or submits a bounded metrics query:

introspection conversations list --lookback 24h --page-all
 
introspection events list \
  --event-name introspection.observation \
  --filter lens=task_resolution \
  --lookback 7d
 
introspection metrics query @metrics.json

For bulk downloads, add --format arrow --arrow-out <path>. Event Arrow pages preserve the family-typed payload struct. --page-all follows opaque cursors to exhaustion; --page-limit bounds the number of pages.

metrics query forwards the JSON document unchanged. A complete count request looks like:

{
  "view": "observations",
  "metrics": [{ "aggregation": "count" }],
  "filters": [
    { "field": "runtime_group_id", "operator": "eq", "value": "019..." }
  ],
  "from_timestamp": "2026-07-11T00:00:00Z",
  "to_timestamp": "2026-07-18T00:00:00Z"
}

Structured output

Data commands accept global output and projection flags:

introspection runtimes list -o table
introspection recipes list --query "[].id"
introspection bindings endpoints list \
  --query "[?kind=='mcp'].{id:id,name:name,url:base_url}" \
  -o table

• -o / --output: json (default), jsonc, or table.
• --query: a JMESPath  expression applied before formatting.

Structured results go to stdout; progress and confirmations go to stderr, so JSON remains safe to pipe.

Machine contract

• Success is exit 0; clap usage errors are exit 2, CLI pre-request read validation uses exit 3, and runtime/API failures use exit 1.
• JSON is the default. --query is JMESPath and still emits JSON; use jq -r when a shell command needs an unquoted scalar.
• List commands return an array for the fetched records. Follow next/--next, or use --page-all where supported; never infer exhaustiveness from one page.
• Progress is stderr. Only stdout should be persisted or piped as structured data.
• Task streams are replayable only within a bounded window. Treat resume_gap as incomplete history and refetch durable task/conversation state.

Upgrade

introspection upgrade
introspection upgrade --check
introspection upgrade --dismiss

The CLI checks the npm latest tag in the background, throttled to roughly once every 20 hours. Notices go to stderr and never corrupt structured output.

Related

• Recipes: the immutable, Git-backed behavior the CLI validates.
• Runtimes: runtime groups, versions, and lane routing.
• Bindings: endpoints, variables, and credentials.
• Tasks: run, cancel, resume, and stream semantics.
• Authentication: CLI login versus SDK API keys.