This post argues that a tool for an agent is a different kind of software artifact from a function or
an API, and should be designed accordingly. Where conventional software establishes a contract between
deterministic systems — the post’s example is that a getWeather call for a given city fetches the
weather the same way every time — a tool is a contract between a deterministic system and a
non-deterministic one. Asked whether to bring an umbrella, an agent might call the weather tool, answer
from general knowledge, ask a clarifying question about location first, or hallucinate or fail to grasp
how to use the tool. The stated consequence is that tools and
Model Context Protocol servers should not be written the way functions and APIs are
written for other developers or systems — they need designing for agents instead.
The method the post proposes is evaluation-driven and collaborative: prototype the tools and use them yourself, build an evaluation from realistic tasks, then paste the evaluation transcripts into Claude Code and let it analyse them and refactor the tools. The authors state that most of the advice in the post came out of repeatedly optimizing Anthropic’s own internal tools this way, against evaluations built on its internal workspace, and that held-out test sets revealed further performance beyond what “expert” implementations achieved — whether those were written manually by the researchers or generated by Claude itself.
The second half distils five principles: choosing which tools to build, namespacing them, returning meaningful context, optimizing responses for token efficiency, and prompt-engineering the descriptions.
Key Points
- A tool is framed as a contract between a deterministic system and a non-deterministic agent, which the post gives as the reason agent tools need designing differently from functions and APIs written for other developers or systems.
- The stated goal is to increase the surface area over which agents can be effective in solving a wide range of tasks, and the authors report that the tools most ergonomic for agents turn out to be surprisingly intuitive for humans to grasp as well.
- Evaluation tasks should be grounded in real use and realistic data sources rather than simplistic sandbox environments; the post says strong tasks might require multiple tool calls, potentially dozens. Its strong examples include preparing a retention offer for a named cancelling customer — working out why they are leaving, what offer would be compelling and what risk factors apply — against weak ones such as looking up a cancellation request by customer ID.
- Each evaluation prompt should be paired with a verifiable response or outcome, ranging from exact string comparison to enlisting Claude as a judge; the post warns against overly strict verifiers that reject correct answers over formatting, punctuation or valid alternative phrasing, and against overspecifying the tools an agent is expected to call, since multiple valid paths may exist.
- Evaluations are recommended to run programmatically as simple agentic loops — a while-loop alternating LLM API and tool calls, one loop per task — with agents instructed to emit reasoning and feedback blocks before their tool calls, which the post suggests may raise effective intelligence by triggering chain-of-thought behaviour.
- Beyond top-level accuracy the post recommends collecting tool-call and task runtime, total tool calls, token consumption and tool errors, and reads patterns in them as design signals: redundant calls suggest rightsizing pagination or token-limit parameters, invalid-parameter errors suggest clearer descriptions or better examples.
- A caution is raised about relying on agent feedback: what agents omit can matter more than what they include, and the post directs readers to the raw transcripts — tool calls and responses — rather than the agents’ stated reasoning alone, noting that evaluation agents do not necessarily know the correct answers themselves.
- More tools are said not to always lead to better outcomes, and a common error named is tools that merely wrap existing software functionality or API endpoints. The reason given is that agents have different affordances from traditional software: an agent has limited context, so a tool returning every contact forces it to read through irrelevant ones, where the natural approach is to skip to the relevant entry.
- The recommended alternative is a few thoughtful tools targeting high-impact workflows, and tools that consolidate several underlying operations — a scheduling tool that finds availability and books the event rather than separate user-listing, event-listing and event-creation tools; a log search tool returning only relevant lines with surrounding context rather than a raw log reader; a single customer-context tool rather than three separate lookups.
- Tool Namespacing — grouping related tools under common prefixes, by service or by resource — is offered as a way to keep boundaries clear when an agent has access to dozens of MCP servers and hundreds of tools. The post reports that the choice between prefix- and suffix-based namespacing had non-trivial effects on its tool-use evaluations, that effects vary by LLM, and recommends deciding by evaluation rather than by rule.
- Tool responses should return only high-signal information and prioritise contextual relevance over flexibility, avoiding low-level technical identifiers in favour of fields more likely to inform an agent’s next action. The authors report that simply resolving arbitrary alphanumeric UUIDs into semantically meaningful language, or even a 0-indexed ID scheme, significantly improved Claude’s precision on retrieval tasks by reducing hallucinations.
- Where both natural-language and technical outputs are needed, the post suggests exposing a response format parameter letting the agent choose concise or detailed responses; in its Slack example the detailed response runs 206 tokens against the concise response’s 72, and concise responses omit the identifiers needed for follow-up calls.
- Response structure itself — XML, JSON or Markdown — is said to affect evaluation performance, with no one-size-fits-all answer, on the reasoning that models trained on next-token prediction tend to do better with formats matching their training data.
- For token efficiency the post recommends pagination, range selection, filtering and truncation with sensible defaults, and states that Claude Code restricts tool responses to 25,000 tokens by default. Truncation and error responses are treated as places to steer the agent, with error text written to communicate specific actionable improvements rather than opaque error codes or tracebacks.
- Prompt-engineering tool descriptions and specs is named as one of the most effective methods for improving tools, with the recommended stance being to describe a tool as you would to a new hire on your team — making implicit context explicit — and to name input parameters unambiguously.
- Two concrete results are reported for description changes: Claude Sonnet 3.5 achieving state-of-the-art performance on SWE-Bench Verified after precise refinements to tool descriptions, which the post says dramatically reduced error rates; and a fix after the team found Claude needlessly appending a year to the web search tool’s query parameter, biasing results and degrading performance.
Context
The post’s evidence is Anthropic’s own internal tooling — Slack and Asana MCP servers, evaluated on its internal workspace — comparing human-written implementations against Claude-optimized ones on held-out test sets. Those comparisons appear as charts without figures in the text, so the size of the reported improvements is not established here; what the text states is the direction and the fact that held-out sets were used to guard against overfitting to the training evaluations.
Its recommendations are stated as practices that worked for the authors rather than as measured findings, with two exceptions that carry specific outcomes: the SWE-bench Verified result attributed to tool-description refinement, and the UUID-resolution effect on retrieval precision. Even the namespacing guidance is deliberately left open, with the post declining to recommend prefix or suffix and telling readers to choose according to their own evaluations.
The closing framing is that software development practice has to re-orient from predictable, deterministic patterns to non-deterministic ones, and that the mechanisms by which agents interact with the world — the MCP protocol, the underlying models — will keep changing, so it is the evaluation-driven loop rather than any particular tool design that the authors present as durable.