Skip to Content
Platform
PlatformLLM providers

LLM providers

Every task needs a model to call. Introspection either manages that access for you through a provider gateway, or routes to your own provider keys when you bring them.

A runtime uses one of two model-access paths. In managed mode, Introspection routes supported model calls through its provider gateway and supplies the upstream credentials. In byok mode, the runtime uses your provider account through an applicable LLM endpoint binding.

LLM mode

Set llm_mode on the runtime to choose how it reaches a model:

  • managed: the task reuses its short-lived Introspection session credential for model access. The protected egress path validates the session and routes supported calls through the managed gateway. No separate per-task gateway key is created.
  • byok: the task uses your provider account. The protected egress path applies the secret headers from the selected LLM endpoint.

You set this once on the runtime; everything below follows from it.

Managed access: the provider gateway

In managed mode, model traffic flows through Introspection’s provider gateway rather than reaching providers directly.

CapabilityWhat managed access provides
Provider-key isolationProvider credentials live in gateway-managed secrets.
Session-bound accessModel access ends with the task session; there is no separate virtual-key lifecycle.
Usage attributionGateway telemetry records provider, model, token usage, and organization, project, and member or session attribution for monitoring and billing.

Managed access does not currently promise a per-task or per-user USD budget. Platform limits and billing controls are separate from model access.

Managed provider families

Managed routing currently covers OpenAI, Anthropic, Gemini, and OpenRouter. Recipes keep Pi’s <provider>/<model> selectors:

anthropic/claude-opus-4-6
openai/gpt-5.4
google/gemini-3-flash-preview
openrouter/google/gemini-3-flash-preview

The gateway routes the model identifier expected by the upstream provider; it does not translate selectors into a separate provider:model format. For a provider outside the managed set, use its OpenRouter model ID or choose BYOK.

Bring-your-own-key (BYOK)

In BYOK mode, your organization supplies the provider credentials through an LLM endpoint binding. Calls use the provider’s native API and bypass the managed gateway; the egress boundary applies the credential.

BYOK usage is still reported through task telemetry, but it is not metered by the managed gateway.

Monitoring usage and cost with token, spend, model, cache, request volume, and latency charts
Monitoring brings model usage, cost, cache behavior, and latency together for the selected services and environment.

Endpoint bindings in BYOK mode

Endpoint bindings select the LLM upstream only in byok mode. A BYOK runtime must have an applicable kind: llm endpoint for its project, runtime group, runtime, and environment. Header values are never returned by the API; the egress layer applies them to outbound requests.

Managed runtimes use the platform gateway instead. Their ordinary api and mcp endpoint bindings still apply.

Real provider credentials are applied only inside the trusted gateway or egress layer, never handed to the agent. Prompts, tool output, and the sandbox filesystem can’t exfiltrate a secret that was never there.

Choosing a path

  • Use managed when Introspection should hold provider keys and route supported models.
  • Use BYOK when calls must use your provider account or an upstream outside the managed set.
  • Platform: the unified map for source control, tools, models, and protected execution.
  • Bindings: BYOK model endpoints and other environment-specific resources.
  • Runtimes: where llm_mode is set on a deployable agent version.
Last updated on