Concepts
A short list of terms covers the whole product. Each links to the page that uses it.
Build
| Term | Meaning |
|---|---|
| Recipe | The files in Git that define an agent: instructions, skills, tools, MCP servers, judges. The same recipe runs on your machine and in Introspection. Create a recipe |
| Agent | A role in a recipe with a model, instructions, tools, and skills. A recipe can hold several. Agents |
| Pi | The open-source agent harness that runs a recipe, locally and in each task’s sandbox. pi.dev |
| Recipes | The Pi package that adds recipe support to Pi. introspection setup installs it. |
| Skill | A recipe skill tells your agent how to do part of its job (Skills). A workflow skill tells your coding agent how to work with Introspection (Using a coding agent). |
Run
| Term | Meaning |
|---|---|
| Runtime | The named, deployable agent built from a recipe. In the API, a “runtime group” (runtime_group_id). Deploy your agent |
| Version | One immutable build of a runtime from one commit. In the API, runtime_id is a version ID. |
| Environment | development, staging, or production. Development runs the files on your machine through introspection dev. Staging and production follow the default branch unless pinned. Test before production |
| Pin and yank | Pin fixes the version an environment serves; yank takes a version out of service. Roll back a release |
| Binding | An endpoint, variable, or credential a runtime gets per environment. The agent can use a credential but never reads it. Bindings |
| Connection | An endpoint in Bindings: a model provider, MCP server, or connector. A user connection is one end user’s authorized account on a connector. |
| Task | One unit of agent work, run in its own sandbox. |
| Run | One turn of a task. |
| Conversation | The saved transcript of a task, with the same ID. Find a conversation |
| Sandbox | The isolated environment for one task. Its filesystem is temporary. The sandbox |
| Runner | The SDK session your app opens against a runtime to start and read tasks. Run tasks from your backend |
| Member | A person or agent in your organization. Projects & members |
| End user | Your customer’s user that a task acts for (identity.user_id in the SDK, --subject in the CLI). The first task for one creates a customer member. How agents act |
Improve
| Term | Meaning |
|---|---|
| Trace | The OpenTelemetry record of one turn: model calls, tool calls, and subagents as spans. |
| Observation and pattern | An observation notes behavior in one conversation; a pattern groups observations that recur. Recurring problems |
| Annotation and dataset | Labels and comments on steps of conversations, grouped by tag. Label production steps |
| Judge and judgement | A recipe-owned quality check, and one result of it. Calibrate a judge |
| Eval | A suite of cases replayed against a change before you ship it. Run and compare evals |
| Experiment | Live traffic split between versions of one runtime, compared with a judge. Run an experiment |
| Control plane and data plane | Where configuration and identity live, and where agents run and their records are stored. Choose where your data runs |
Find it in the dashboard
| Sidebar | What you do there |
|---|---|
| Runtimes | See versions, pin, roll back, yank. Ship |
| Bindings | Connect providers, MCP servers, and connectors; set variables and secrets. Bindings |
| Repositories | Register repositories agents may clone and change. Repositories |
| Conversations | Read any task end to end. Conversations |
| Events, Patterns | Feedback, observations, judgements, recurring behavior. Patterns |
| Metrics | Usage, reliability, and cost. Metrics |
| More → Files, Users | Outputs and memory; your end users. Files |
| Datasets | Label production examples as a team. Annotate |
| Judges | Pass rates and live settings. Judges |
| Experiments | Compare versions on live traffic (shown once a judge exists). Experiments |
| Settings | Project, Connectors, Applications, API keys. Members and integrations are organization settings. Identity & access |
Last updated on