Observations & Patterns

These three layers take everything your agent emits and surface it as something you can filter, count, and trend, so you start from “this happened 240 times this week” and drill down rather than scrolling logs. Each layer builds on the one below it.

Events

An event is a single durable telemetry record. Events are the unified research surface: they include the raw activity your runtime emits over the log stream, plus the platform-generated introspection.* records (observations, patterns, judgements, and feedback) that the platform writes back onto the same stream.

Because everything lands on one stream, the behavior you research and the behavior you measure share the same shape and lineage. Events are append-only and immutable: written once and never changed, so any view you build over them is reproducible.

Reading events: one family per request

GET /v1/events serves six canonical, typed introspection.* families — feedback, observation, observation_clustering.run, judgement, pattern, and pattern.assignment — and every list read names exactly one of them via the required event_name parameter. A response is therefore always homogeneous: each row carries a common envelope (id, timestamp, event_name, trace/span ids, conversation, service, environment, lineage) plus a nested payload typed to the requested family.

The families split into two kinds:

• State families (introspection.observation, introspection.pattern): the server folds the event history and returns current state — an observation row has supersession applied and carries its current pattern assignment; a pattern row is the folded catalog entry with its current name, status, and lifecycle timestamps.
• Stream families (introspection.feedback, introspection.judgement, introspection.pattern.assignment, introspection.observation_clustering.run): plain event rows — one thing that happened.

“A pattern’s observations” is composed with the family filter: event_name=introspection.observation&pattern_id=X returns the resolved observations currently assigned to that pattern, and event_name=introspection.pattern returns the catalog rows. Rows outside the six families — custom track() events, gen_ai.* records — are not enumerable through this read; they remain fully measurable through the metrics API below.

Measuring events: the metrics API

Retrieval and aggregation are separate surfaces. GET /v1/events retrieves rows of one named family; counting, trending, and slicing happen through POST /v1/metrics, a single bounded query endpoint over the telemetry stores.

A metrics query names a view (spans, conversations, events, judgements, observations, or patterns), one to six metrics (an aggregation such as count, count_distinct, sum, avg, min/max, or a percentile p50–p99 over an allow-listed measure), up to four dimensions to group by, filters, an optional time dimension for bucketed timeseries (pass granularity: "auto" to let the platform pick the finest interval that fits), post-aggregation having conditions, and a series_limit that ranks the top series over the whole window and returns them complete.

Every field you can measure, group, or filter on is allow-listed per view — the API accepts no SQL and no open-ended queries, and every query is capped (window length, buckets, rows, series), so a dashboard can never turn into an unbounded scan. Typical uses: observation counts per lens per day, judge pass rates per runtime version, p95 conversation duration per experiment arm, or user feedback trends (events view, filtered to event_name = introspection.feedback, grouped by sentiment).

See the API Reference for the request grammar and per-view field lists.

Observations

An observation is something the platform noticed about one completed conversation. It is generated automatically when a conversation completes: there’s nothing to instrument and nothing to enable.

Every observation is recorded along exactly one of five lenses:

A single conversation produces between zero and five observations: at most one per lens, and only when there’s something worth noting. Each observation carries a short label, a one-sentence summary, a severity, a confidence, and references back into the conversation as evidence.

Because observations are generated per completed conversation, a long-running conversation that completes, continues, and completes again is observed in segments: later turns become new observations rather than re-stating earlier ones. Trends are counted over distinct conversations so repeats don’t inflate them.

Patterns

A pattern is a named cluster of similar observations across many conversations. Where an observation is about one conversation, a pattern is about a recurring behavior or failure mode: it’s how “the agent keeps doing X” becomes a single thing you can point at, name, and track.

Patterns are mutable: as new observations arrive they’re sorted into existing patterns, and the cluster map is periodically regenerated, naming patterns, refining descriptions, minting new ones, and retiring stale ones. A pattern keeps a stable identity across regenerations even as its name and membership drift, so its trend line stays continuous.

Patterns are scoped per organization, project, runtime group, and lens. A pattern means “this recurring behavior, for this agent lineage, on this lens,” so the count you read is always about a coherent population rather than a mix of unrelated agents.

The investigation path

Investigation runs top-down:

Patterns          → "agent_struggle: re-asks for confirmation it already has" (240×)
  → Observations  → the individual conversations that landed in this cluster
    → Conversations → the full, replayable transcript of any one of them

You start from a pattern (the aggregate behavior), drill into the observations that make it up, and open the underlying conversation replay to see exactly what happened. Each step narrows from “what is my agent doing at scale” to “show me the turn where it went wrong.”

Reproducible by design

Investigation gives you reproducible, filterable answers about agent behavior: “how many conversations hit this failure mode this week, broken down by version” returns the same structured result every time you ask, not a different ranked list.

Structured lenses and patterns mean the same query gives the same answer, you can filter and group on stable dimensions, and you can trend a behavior across runtime versions with confidence. That reproducibility is what lets observations feed directly into judges and experiments without re-litigating what you were measuring.

How it connects

• Conversations are the source: an observation is generated when a conversation completes, and the investigation path bottoms out in the conversation replay.
• Judges complement observations: observations are what the platform noticed automatically; a judge is what you decide to measure. Both live on the same event stream.