Agent development lifecycle

This guide follows the refund-exception-agent after its first deployment. The change is small but meaningful: final-sale orders must be declined or escalated, never promised a refund.

What each stage proves

This lifecycle begins after the Quickstart creates the first runtime. The initial runtime version activates both staging and production; later commits create candidates that can be verified in staging before merge.

1. Prove the change locally

Edit the policy skill or agent instructions in the recipe repository. Then run the changed behavior and nearby control cases through fresh Pi sessions:

introspection local --runtime refund-exception-agent \
  --print "Order 1842 is final-sale and inside 30 days. Decide eligibility and draft the next response."

The expected decision is decline or escalate, depending on the policy. The response must not promise a refund. Also run an ordinary eligible refund and an ambiguous case so the fix does not become a blanket denial.

Use the coding agent driving the change to make the checkpoint explicit:

Run the final-sale case, one eligible refund, and one ambiguous case through
fresh local sessions. Show the decision for each case and do not edit again
until you can explain any failure.

Expected result: the intended case changes, the controls remain correct, and introspection check passes.

The local recipe reference covers recipe selection, structured output, and forwarding arguments to Pi.

2. Test through the application path

Local Pi proves recipe behavior. Development proves the integration: identity, bindings, streaming, files, reconnect behavior, and the UI that presents the answer.

Attach the working tree to the development runtime:

introspection dev --runtime refund-exception-agent

Keep that process running and send the same final-sale case through your application with a development-scoped credential. The SDK call does not take an environment option; the credential selects development.

const runner = await client.runtimes("refund-exception-agent").run({
  identity: { user_id: "development-check" },
});
 
const task = await runner.tasks.start({
  prompt: "Order 1842 is final-sale and inside 30 days. Decide eligibility and draft the next response.",
});
 
console.log(await task.text());
await runner.close();
await client.shutdown();

Expected result: the application shows the corrected decision, the task is owned by development-check, and the resulting conversation identifies the development runtime.

Recipe reload boundaries, multiple developer attachments, local MCP routing, and INTROSPECTION_DEV_TARGET belong in the introspection dev reference.

3. Verify the committed candidate in staging

Commit and push the reviewed recipe change. The GitHub integration creates an immutable candidate runtime version from that commit. List the versions and pin staging to the candidate:

introspection runtimes list --runtime refund-exception-agent -o table
introspection runtimes pin <candidate-runtime-id>

Configure any staging bindings the recipe requires, then run the same three cases through the stable runtime slug with a staging-scoped credential. Open the resulting conversation and confirm its runtime version matches the candidate commit.

Expected result: staging resolves the candidate version, every required binding is healthy, and the final-sale and control cases match the local proof.

Do not treat a completed task as sufficient proof. Verify the output, runtime version, identity, tool trajectory, and conversation.

4. Merge and verify production

Merge the pull request into the integrated repository’s production branch. Unpinned production follows that branch; the runtime version itself remains immutable.

Wait until the production lane resolves the merged commit, then run one bounded smoke task with a production-scoped credential. Confirm the same facts you checked in staging:

• exact runtime version and commit;
• expected decision and response;
• correct customer identity;
• healthy bindings and tool calls;
• durable task and conversation evidence.

Expected result: new production tasks use the merged version. Existing tasks keep the version they started with.

Lane assignment, yanking a bad version, and first-runtime behavior are covered in Operating runtimes.

5. Learn from the result

After release, watch the signals tied to this behavior: user corrections, product outcomes, judge failures, and recurring patterns. Open the exact source conversation before proposing another change.

The next cycle starts from a production signal, not from a vague prompt rewrite.

Common failure pattern: proving different things at each stage

Changing the prompt, identity, bindings, or expected outcome between local, development, staging, and production makes the checks incomparable. Keep a small set of named cases and run the same cases at every stage. Add a new case only when it represents a new claim you intend to verify.

Done when

• The changed behavior and nearby controls pass in fresh local sessions.
• The real application path works with development identity and bindings.
• Staging runs the exact committed candidate you reviewed.
• Production resolves the merged version and passes a bounded smoke task.
• The resulting conversation can seed the next continual learning cycle.

Next

• Embed the runtime in a product with Full-stack applications.
• Diagnose a real outcome with Learn from production.
• Use Operating runtimes for lane and rollback procedures.