Usage, cost, and alerts
You need: introspection login, or an API key in CI. For single conversations, see Find a conversation.
The dashboard’s Metrics page charts usage, errors, and cost by runtime and environment. For anything else, write a query.
Write a query
-
Save the question as JSON:
cost-by-user.json{ "view": "conversations", "metrics": [{ "aggregation": "sum", "measure": "total_cost_usd" }], "dimensions": [{ "field": "user_id" }], "filters": [{ "field": "environment", "operator": "eq", "value": "production" }], "from_timestamp": "2026-10-01T00:00:00Z", "to_timestamp": "2026-10-08T00:00:00Z" } -
Run it:
introspection metrics query @cost-by-user.json -o json
Each row of data is one group. Use -o json; a table hides the rows. To print one number, add --query "data[0].metrics[0].value".
Query fields and the conversations view
| Field | Content |
|---|---|
view | spans, conversations, events, judgements, observations, patterns, or files |
metrics | [{aggregation, measure}]: count (no measure), count_distinct, sum, avg, min, max, p50, p75, p90, p95, p99 |
dimensions | Fields to group by |
filters | [{field, operator, value}]: eq, neq, gt, gte, lt, lte, in, nin, exists, contains |
time_dimension | {"bins": N} for a time series |
from_timestamp, to_timestamp | Required |
The conversations view has one row per conversation:
- Dimensions:
environment,runtime_group_id,runtime_id,recipe_git_commit_sha,experiment_id,model(first model),agent_name,user_id,owner_key,service_name,status - Measures:
total_cost_usd,duration_ns,total_tokens,input_tokens,output_tokens,span_count,error_count,has_error
Here has_error is 1 (and status is Error) when any step errored, tool calls included. The conversations list’s status counts only failed model calls.
Other views’ fields are in the API reference. An unknown field returns 400.
Common queries
| Question | View | Metric | Group or filter |
|---|---|---|---|
| Cost per runtime | conversations | sum total_cost_usd | group runtime_group_id |
| Error rate per runtime | conversations | avg has_error | group runtime_group_id |
| Monthly cost per end user | conversations | sum total_cost_usd | group user_id; from_timestamp–to_timestamp span one month |
| Cost by model | spans | sum gen_ai.cost.usd | group gen_ai.request.model |
| Tool error rate by tool | spans | avg status_error | group gen_ai.tool.name; filter gen_ai.operation.name eq execute_tool |
| Judge pass rate per release | judgements | count | group judge_id, judgement_result, recipe_git_commit_sha |
| Conversations in a pattern | observations | count_distinct conversation_id | filter pattern_id eq <id> |
| Negative feedback | events | count | filter event_name eq introspection.feedback; group sentiment |
| A business event | events | count, or sum value | filter event_name eq case_closed (your track() name) |
Alert on a threshold
There are no built-in alerts. Save the query, then have your scheduler or monitoring run it and alert when the value passes your limit. Yesterday’s production spend:
{"view":"conversations","metrics":[{"aggregation":"sum","measure":"total_cost_usd"}],
"filters":[{"field":"environment","operator":"eq","value":"production"}],
"from_timestamp":"2026-11-02T00:00:00Z","to_timestamp":"2026-11-03T00:00:00Z"}introspection metrics query @spend.json --query "data[0].metrics[0].value" -o table-o table prints the single number bare. The query takes absolute times, so the scheduler fills in each day’s dates.
Use an Admin Agent key: a default key sees only its own conversations. For a per-tenant alert, add a user_id or runtime_group_id filter.
Limits
| Not supported | Do this instead |
|---|---|
| Spend caps | Alert as above, and limit traffic in your app. |
| Relative time ranges (“last 24 hours”) | Your scheduler writes the dates into the query. |
| Grouping by task tags or metadata | Use a runtime per customer, or track() an event per entity. |
| Judge cost | Not reported. A byok judge shows on your provider bill. |
Or ask your agent: “What did support-agent cost per end user last week, and which tool fails most?”