# Debug a bad answer

Find the first step that went wrong, check whether it recurs, and name the part of the agent that owns the fix.

Use this when a user reports a wrong answer.

**You need:** `introspection login` and the task ID, or enough detail to [find the conversation](/observe/conversations#filter-the-list).

## 1. Open the conversation

```bash
introspection tasks get <task-id>                                      # version, commit, why it ended
introspection conversations get <task-id> --output-file failure.json
```

If `tasks get` shows an error and there is no conversation, the task failed before the agent ran: see [Troubleshooting](/support/troubleshooting).

## 2. Find the first wrong step

Read forward and stop at the first step that differs from what you expected. Later mistakes are often consequences of it.

```text
Policy read: final-sale items are not refundable          correct
Order read: item 1842 is final-sale                       correct
Decision: approve refund under the 30-day return rule     first wrong step
```

`status: Ok` only means no model call failed. A conversation can be `Ok` and wrong, even with failed tool calls.

## 3. Check how often it happens

Open the conversation's observations, then the pattern they joined, to see other conversations with the same problem:

```bash
introspection events list --event-name introspection.observation --filter conversation_id=<task-id>
introspection events list --event-name introspection.observation --filter pattern_id=<pattern-id> --lookback 30d
```

To get a number, count distinct conversations in the pattern ([Usage, cost, and alerts](/observe/metrics#common-queries)).

## 4. Name the owner

Compare with a few good conversations, then pick the owner:

| First wrong step | Owner | Fix in |
| --- | --- | --- |
| Right evidence, wrong action | Instructions, a skill, or agent settings | [Build](/build/agents) |
| Wrong or missing tool result | Tool code or its data | [Tools](/build/tools) |
| Missing endpoint, credential, or permission | Bindings | [Connect services and secrets](/ship/bindings) |
| The wrong version answered | Deployment | [Roll back a release](/ship/rollback) |
| Nobody agrees what the right answer is | Your product owner | Decide, then write it down as an eval case |

## 5. Keep the evidence

Write one sentence an eval could prove wrong ("the agent applies the 30-day rule to final-sale items"), and keep the failing trace IDs. Turn them into cases: [Add production cases to an eval](/improve/eval-cases).

## Verify

After a fix ships, rerun the failing prompt in staging (`tasks create -e staging`) and confirm the first wrong step is gone.

## Limits

| Not supported | Do this instead |
| --- | --- |
| Replaying a production task with its files and tool results | `introspection eval run -p <suite> --from-conversation <id> --turn <n>` replays the conversation's messages locally, but not its files or tool side effects; fake the tools with [fakes](/improve/fakes). |
| Searching conversations by text | Export, then search locally. |

Or ask your agent: "Find out why task `<task-id>` approved a final-sale refund. Find the first wrong step and how often it happens. Don't change anything."

Debug a bad answer

Find the first step that went wrong, check whether it recurs, and name the part of the agent that owns the fix.

Use this when a user reports a wrong answer.

You need: introspection login and the task ID, or enough detail to find the conversation.

1. Open the conversation


introspection tasks get <task-id>                                      # version, commit, why it ended
introspection conversations get <task-id> --output-file failure.json

If tasks get shows an error and there is no conversation, the task failed before the agent ran: see Troubleshooting.

2. Find the first wrong step

Read forward and stop at the first step that differs from what you expected. Later mistakes are often consequences of it.


Policy read: final-sale items are not refundable          correct
Order read: item 1842 is final-sale                       correct
Decision: approve refund under the 30-day return rule     first wrong step

status: Ok only means no model call failed. A conversation can be Ok and wrong, even with failed tool calls.

3. Check how often it happens

Open the conversation’s observations, then the pattern they joined, to see other conversations with the same problem:


introspection events list --event-name introspection.observation --filter conversation_id=<task-id>
introspection events list --event-name introspection.observation --filter pattern_id=<pattern-id> --lookback 30d

To get a number, count distinct conversations in the pattern (Usage, cost, and alerts).

4. Name the owner

Compare with a few good conversations, then pick the owner:

First wrong step	Owner	Fix in
Right evidence, wrong action	Instructions, a skill, or agent settings	Build
Wrong or missing tool result	Tool code or its data	Tools
Missing endpoint, credential, or permission	Bindings	Connect services and secrets
The wrong version answered	Deployment	Roll back a release
Nobody agrees what the right answer is	Your product owner	Decide, then write it down as an eval case

5. Keep the evidence

Write one sentence an eval could prove wrong (“the agent applies the 30-day rule to final-sale items”), and keep the failing trace IDs. Turn them into cases: Add production cases to an eval.

Verify

After a fix ships, rerun the failing prompt in staging (tasks create -e staging) and confirm the first wrong step is gone.

Limits

Not supported	Do this instead
Replaying a production task with its files and tool results	`introspection eval run -p <suite> --from-conversation <id> --turn <n>` replays the conversation’s messages locally, but not its files or tool side effects; fake the tools with fakes.
Searching conversations by text	Export, then search locally.

Or ask your agent: “Find out why task <task-id> approved a final-sale refund. Find the first wrong step and how often it happens. Don’t change anything.”