Fake MCP servers

A fake stands in for one MCP server your recipe declares while an eval runs. It serves the tools captured from the real server, validates every call against their schemas, and keeps state that your checks read afterwards. A scenario sets the situation the agent meets: the records each fake starts with and the calls that will fail.

You build fakes from your eval cases. Each case says which servers it uses, which tools it needs, which records must exist and which failure it meets, and the fake implements that and nothing more. Your coding agent does this work in the fakes step of the create, improve and optimize workflows, so the usual way to start is to ask it. This page shows the parts it builds, so you can review them.

You need: CLI 0.54.0 or later, @introspection-ai/mcp-fakes 0.2.1 or later (and introspection-harbor 0.10.0 or later under Harbor), and each MCP server bound to your project (Bindings).

1. Capture the real server’s tools

Bind each server to the project yourself, then capture its contract and generate TypeScript types from it:

introspection bindings mcp capture attio --out fakes/mcp
introspection bindings mcp types fakes/mcp/attio/tools.json --out fakes/mcp/attio/contract.ts

Capture reads the server’s identity and its tool list and never calls a tool. It writes tools.json and starts manifest.json. The directory name must be the server id your recipe declares, even when the vendor’s name differs. The MCP tool list is the contract: the vendor’s REST API does not show which fields its MCP server exposes. On your laptop, capture accepts plain http only for loopback and http://host.docker.internal; a binding with a credential must be https.

2. Learn real call shapes

If an agent in your project already calls the server, read how it does:

introspection bindings mcp observe attio --runtime support-agent --lookback 30d

observe writes observed.json: which tools and arguments the agent used, the shape of each response, the errors the server returned, and how paging worked, with every value masked. It reads only calls made to the live server, so answers from a fake never feed back into it. Your handlers then answer in those shapes. Where no traces show a response, the fake writes it from the tool’s description, and your agent names it in its report. Nobody creates a record or sends a message on the real server to find out.

observed.json records each tool’s result format (JSON, TOON, JSON after a prose prefix, or plain text) and the schema of the decoded data. One written before CLI 0.54.0 is refused by --verify: run observe again for each server after upgrading.

--examples writes sampled raw calls to a folder outside your Git tree. They are customer data: read them, but never commit them.

3. Set up the fakes package

fakes/
  package.json          # depends on @introspection-ai/mcp-fakes
  Dockerfile            # the Harbor sidecar image
  dist/sidecar.js       # the bundled sidecar, committed
  mcp/
    index.ts            # export const fakes = { attio, resend }
    attio/
      tools.json        # from capture
      contract.ts       # from types
      observed.json     # from observe
      entities.ts       # the fake's state type
      handlers.ts       # one callback per allowed tool
      manifest.json     # server, capture, entities, tool map
      index.ts          # exports manifest and register
  scenarios/
    index.ts            # SCENARIOS and world(name, clock)
    base.ts             # the world every scenario starts from
    blocked.ts          # one module per scenario
  tests/fakes.test.js   # checkRecipeFakes

fakes/ is its own workspace package. Add it to pnpm-workspace.yaml (packages: [fakes]), include fakes/ in your tsconfig.json, and add fakes/ to .recipeignore, since a fake is never deployed:

{
  "name": "fakes",
  "private": true,
  "type": "module",
  "scripts": { "test": "node --test tests/*.test.js", "bundle": "mcp-fakes bundle" },
  "dependencies": { "@introspection-ai/mcp-fakes": "^0.2.1", "@toon-format/toon": "2.2.0" }
}

4. Describe what the server stores

A fake starts from the things the server stores, such as companies and deals, written as types in entities.ts. The manifest says which tools read and write each of them:

{
  "server": { "name": "attio", "version": "0.1.0" },
  "capture": { "at": "2026-10-15T09:00:00Z", "endpoint": "https://mcp.attio.com/mcp", "hash": "sha256:4f1c…" },
  "entities": ["companies", "deals"],
  "tools": {
    "get-records-by-ids": { "kind": "stateful", "reads": ["companies", "deals"], "writes": [] },
    "list-records": { "kind": "stateful", "reads": ["companies", "deals"], "writes": [] },
    "update-record": { "kind": "stateful", "reads": ["deals"], "writes": ["deals"] }
  }
}

Capture writes server and capture, and keeps the rest when it runs again. tools lists exactly the tools your recipe allows. A tool is stateful when it reads or changes stored records, so a deal updated by one call shows up in the next read. A tool may be static, always giving the same answer, only for records no tool writes. The write map must agree with each tool’s readOnlyHint annotation.

5. Write handlers

Each handler is an ordinary MCP SDK tool callback, registered in tools.json order with the tool’s captured schema. Here is the one that changes a deal:

import { type McpServer, notImplemented, text, toolConfig } from "@introspection-ai/mcp-fakes";
import { encode } from "@toon-format/toon";
import type { Contract } from "./contract.ts";
import type { State } from "./entities.ts";
import tools from "./tools.json" with { type: "json" };
 
const config = toolConfig<Contract>(tools);
 
export function register(server: McpServer, state: State) {
  server.registerTool("update-record", config("update-record"), async ({ object, record_id, values }) => {
    if (object !== "deals") return notImplemented(`updating ${object}`);
    const deal = state.deals.find((item) => item.id === record_id);
    if (!deal) return text(`The record with ID "${record_id}" was not found.`, true);
    deal.stage = String(values.stage);
    return text(encode({ record_id: deal.id })); // Attio answers in TOON
  });
  // ...one registerTool per tool in the manifest
}

A handler answers in the real server’s form. Attio answers text in TOON, so the fake builds the result as data and encodes it with @toon-format/toon rather than writing the text by hand. A server that answers JSON text gets JSON.stringify, and structured(...) is only for a tool that declares an output schema. An argument value the handler does not implement gets notImplemented(...), a tool error, instead of a plausible answer. The fake enforces what the server enforces and not your recipe’s rules, so a mail fake accepts a send your recipe should have asked approval for, and your checks catch the mistake.

6. Write scenarios

A scenario is a module that changes the base world. It sets state, including failure conditions the handlers read, and never defines behavior:

import { labelId } from "@introspection-ai/mcp-fakes";
import type { World } from "./index.ts";
 
// Updating the Acme deal is rejected by the CRM.
export default function blocked(world: World) {
  world.attio.failUpdate = labelId("acme-growth");
}

scenarios/index.ts exports SCENARIOS and world(name, clock), which builds the base world when no name is given and throws on a name it does not know, so a misspelt scenario never runs as the base world. Records are named by readable labels, labelId turns a label into a stable id, every date is computed from the clock, and all data is fictional.

7. Bind the fakes in a Vitest case and check the result

import { fileURLToPath } from "node:url";
import { type Services, startServices } from "@introspection-ai/mcp-fakes";
import { loadManifests, writeAttempts } from "@introspection-ai/mcp-fakes/verify";
import { createJudge, describeEval } from "vitest-evals";
import { createRecipeHarness, recipeCases, type RecipeEvalInput } from "@introspection-ai/vitest-evals";
import { fakes } from "../../fakes/mcp/index.ts";
import { type World, world } from "../../fakes/scenarios/index.ts";
 
const manifests = loadManifests(fileURLToPath(new URL("../../fakes/mcp", import.meta.url)));
 
const NoCrmWrite = createJudge("no-crm-write", ({ toolCalls }) => ({
  score: writeAttempts(toolCalls ?? [], { attio: manifests.attio }).length === 0 ? 1 : 0,
}));
 
const harness = createRecipeHarness<RecipeEvalInput & { scenario?: string }, Services<World>>({
  beforeRun: async ({ input }) => {
    const services = await startServices(fakes, { world, scenario: input.scenario });
    return { mcpServers: services.mcpServers(), fixture: services, teardown: services.close };
  },
  collectArtifacts: ({ fixture }) => ({ fakeStates: { initial: fixture.initial, final: structuredClone(fixture.states) } }),
});
 
describeEval("CRM read-only questions", { harness, judges: [NoCrmWrite] }, (it) => {
  it.for(recipeCases("crm"))("$name", async ({ input }, { run }) => { await run(input); });
});

Each session gets a fresh set of fakes on a free local port, built from the case’s scenario, or the base world when the case names none. writeAttempts returns the calls to tools that write, which makes it an attempt check. To check state instead, changedEntities(states, manifests) compares the fakeStates artifact’s initial and final state and returns the written entities that changed, named like attio.deals. Pass clock to fix “today” for a case that depends on dates.

8. Run them in Harbor

Under Harbor all fakes run in one sidecar on port 7353. Run pnpm exec mcp-fakes bundle in fakes/ and commit dist/sidecar.js. The image copies fakes/ and runs that file, so it installs nothing. The task binds each server to http://fakes:7353/<server-id>/mcp, picks its scenario with MCP_FAKES_SCENARIO, and its verifier checks state with introspection_harbor.fakes. See Harbor.

Check the fakes

Every check in this table runs offline, so you can add each one to CI:

Two checks compare against the live server and your project’s traces. They need a teammate’s introspection login, because a project API key sees only its own calls, so run them on a schedule from a teammate’s machine rather than in CI:

When the server changed, capture again, regenerate the types, rerun these checks, and rerun your evals.

Limits

Or ask your agent: “Build MCP fakes for our attio server and use them in the crm eval.”