LLM providers
A runtime doesn’t hard-code where its model lives. That’s the job of bindings and the runtime’s LLM mode. There are two ways a task reaches a model: Introspection-managed access through a provider gateway, or bring-your-own-key (BYOK) straight to the provider. A runtime’s llm_mode (managed or byok) decides which.
LLM mode
llm_mode is one of the two delivery properties you set on a runtime (alongside kind):
managed: the runtime uses Introspection-managed model access. The platform holds the provider keys; your task gets a scoped virtual key instead.byok: the runtime uses your own provider credentials. Calls go to the provider directly, bypassing the provider gateway (your key is still injected at the egress boundary, never handed to the agent).
You set this once on the runtime; everything below follows from it.
Managed access: the provider gateway
In managed mode, model traffic flows through a self-hosted provider gateway (proxy.introspection.dev) rather than reaching providers directly. The gateway is the preferred path for platform LLM traffic, and it does three things the per-session credential-injection path structurally can’t:
| Capability | What the gateway adds |
|---|---|
| Virtual keys | A per-task key replaces per-session credential maps. The real provider key never enters the sandbox. |
| Budgets | A USD spending cap can be assigned per gateway user, enforced per request. Over budget returns 402 Payment Required. |
| Usage tracking | Every request is logged with exact provider, model, token counts, and computed cost, attributable to a specific org, member, project, and task. |
Virtual keys
Each managed task is issued a virtual key scoped to a stable gateway user (derived from the org and billing member). The key is named per task, carries org/project/member/task metadata for attribution, expires after 24h by default (extended while the task runs), and is revoked on completion. The sandbox receives the virtual key and a gateway base URL, never the platform’s real provider key.
Managed provider families
Platform-managed routing covers four provider families: OpenAI, Anthropic, Gemini, and OpenRouter as the catch-all for long-tail and open-source models. Recipe model selectors stay in Pi’s <provider>/<model> form and are translated to the gateway’s provider:model form at the boundary:
anthropic/claude-opus-4-6 -> anthropic:claude-opus-4-6
openai/gpt-5.4 -> openai:gpt-5.4
google/gemini-3-flash -> gemini:gemini-3-flash
openrouter/google/gemini-… -> openrouter:google/gemini-…Managed mode rejects providers outside those four families rather than guessing: a model like Mistral must be selected as an OpenRouter ID (openrouter/mistralai/devstral-…). BYOK mode keeps support for the full Pi catalog.
Bring-your-own-key (BYOK)
In BYOK mode, the org supplies its own provider API keys and the sandbox calls providers at their native base URL, bypassing the provider gateway. The credential is still injected at the egress boundary rather than exposed to the agent.
The trade-off is explicit: BYOK traffic does not go through the gateway, so it gets no virtual-key usage tracking and no budget enforcement, and it doesn’t need the provider:model prefix. Usage for BYOK is still reported, but from telemetry rather than the gateway’s per-request log.
Endpoints: a runtime’s upstream model target
Whichever mode a runtime is in, the concrete upstream it talks to is an endpoint, most often a kind: llm model gateway. An endpoint stores a canonical host, an optional base_url, and a set of encrypted headers that hold the credential.
The security boundary is the same one bindings describe everywhere: header values are never returned by the API and never reach the sandbox. The agent learns which hosts and base URLs it may reach; the egress layer applies the secret headers on the way out. Like all bindings, an endpoint is scoped by project, runtime group, runtime, or environment, so a staging lane can point at a sandbox gateway while production points at the real one, without touching the recipe.
Real provider credentials, whether the gateway’s keys or your BYOK keys, are applied at the egress boundary, not handed to the agent. Prompts, tool output, and the sandbox filesystem can’t exfiltrate a secret that was never there.
Choosing a path
- Reach for managed when you want the platform to hold keys, enforce budgets, and give you exact per-task cost attribution out of the box. This is the default for most runtimes.
- Reach for BYOK when you must use your own provider account or need a model outside the four managed families, and you’re prepared to manage keys and usage yourself.