Tool Selection: Writing Descriptions That Stop Wrong Tool Calls

Why the model picks the wrong tool

An agent has two entirely different ways to fail at a tool call, and teams routinely apply the fix for one to the symptoms of the other. The first is a selection failure: the model reaches for the wrong tool, reaches for a tool when it should have answered directly, or answers directly when it should have called something. The second is an execution failure: the model picks correctly and then supplies a malformed argument, or the tool returns something the agent cannot recover from. Both look identical in a support ticket, and they have almost nothing in common underneath.

Execution failures are fixed on the schema side, with typed parameters, structured error envelopes, idempotent writes and bounded retries. That work has its own playbook, and this guide deliberately does not repeat it: our companion piece on designing tools for AI agents — schemas, errors and retries covers the mechanics of what happens once a tool has been chosen. This guide is about the step before that, and it is fixed elsewhere entirely: in the words. The tool's name, its description, and the descriptions on its parameters are what the model reads when it decides which tool to reach for. Division of labour: that guide owns the contract, this one owns the copy.

The uncomfortable part is that in most codebases those words were written once, quickly, by whoever added the tool, and never reviewed since — yet they ship into every single request the agent handles. When a model picks wrongly, the instinct is to patch the system prompt or upgrade the model. The cheaper fix is nearly always fifteen words in a description.

• Selection and execution are different bugs. Read the trace and classify before changing anything.
• Descriptions are prompt surface, not API documentation. Write them for the model as the reader.
• Six failure modes cover nearly all of it, each with a signature you can spot in a trace.
• The fix pattern is stable: capability line, use-this-when, do-not-use-this-when, units, worked example.
• Near-duplicate tools are the hard case, with four legitimate responses — only one of which is rewriting text.
• Selection is measurable. A labelled case set plus a confusion matrix turns opinion into a decision.

The six ways tool selection fails

Diagnosis first. Every selection bug I have chased down has fitted one of six shapes, and naming the shape tells you which fix to apply. The trap is treating all six as "the description is not clear enough", because two are not description problems at all.

The first four are genuine description problems, though near-duplicates often need more than a rewrite; they get their own section below. The fifth is worth separating because the metrics look fine — the model chose correctly, so a selection eval scores it as a pass while the user gets a wrong answer. The sixth is the most misread.

Position effects deserve a moment. If shuffling the order of your tool list changes which tool gets called for the same request, the reflex is to find the best ordering and lock it in. Resist that: the ordering did not cause the problem, it revealed one. Sensitivity to position means two or more candidates look similar enough that near-arbitrary factors decide between them, and the moment you add a seventh tool your careful ordering stops helping. Use the reorder as a diagnostic — shuffle, re-run your cases, and treat every tool whose accuracy moves as a description that needs work.

Anatomy of a description that works

There is a pattern here that survives model upgrades and provider changes, because it removes ambiguity rather than exploiting any particular model's habits. Five parts, in this order.

The five parts

One capability line. What the tool does, in one sentence, in the vocabulary a user would use rather than your internal one. It should read like the answer to "what is this for", not an endpoint name expanded into a sentence.

An explicit use-this-when clause. This is the part most descriptions are missing, and it is the difference between a tool that fires on obvious phrasings and one that fires on real ones. Users do not say "search the help articles"; they say "why was I charged twice". Name the situations, not just the capability.

An explicit do-not-use-this-when clause, with a redirect. Boundaries are as informative as capabilities, and one that names the sibling tool is worth several paragraphs of careful positive phrasing. "Do not use this for a specific customer's orders — use search_orders" resolves an ambiguity in eleven words.

Units, formats and coverage on anything guessable. Some of this belongs on the parameters, but the tool-level description is the right home for coverage limits: what data the tool can and cannot see, which regions it serves, whether it reads live or cached state.

One worked example call. A single realistic invocation, inline in the description text, does more to pin down argument shape than another sentence of explanation. It costs one line.

Before and after

Here is a weak description and its rewrite. The original is not a straw man — it is the shape most tools ship with, written as a note-to-self by the engineer who wired up the endpoint.

# BEFORE — four words, three problems
{
    "name": "search",
    "description": "Searches the knowledge base.",
}

# Problem 1: the name is a category, not an action. Any request
#            containing "find" or "look up" is a candidate.
# Problem 2: "knowledge base" is internal vocabulary. Users say
#            "help page", "FAQ", "the docs" - none of which match.
# Problem 3: no boundary. Nothing here says this tool cannot see
#            a specific customer's account, so it gets called for
#            "find my last order" and returns nothing useful.


# AFTER — capability, when, when-not, coverage, example
{
    "name": "search_help_articles",
    "description": (
        "Search published customer-facing help articles by "
        "natural-language query and return the top matches with "
        "title, URL and a short excerpt. "
        "USE THIS WHEN the user asks how to do something, what a "
        "policy says, or why a feature behaves the way it does - "
        "including indirect phrasings such as 'why was I charged "
        "twice' or 'how long do I have to return this'. "
        "DO NOT USE THIS to look up a specific customer's orders, "
        "payments or account data - use search_orders or "
        "get_account_summary instead. "
        "Covers only articles published in the public help center; "
        "it cannot see internal runbooks or unpublished drafts. "
        "Example: search_help_articles(query='return window for "
        "damaged items', locale='en-GB', limit=5)."
    ),
}

Five sentences, and every one changes a decision. The name now contains a verb and a noun, so it no longer competes for every request containing the word "find". The use-this-when clause carries two indirect phrasings, which converts a tool that fires on explicit search requests into one that fires on real ones. The boundary clause names its two siblings, so the model has somewhere to go instead of guessing, and the coverage sentence closes off a whole class of wrong calls.

Note what is not in there: no explanation of the retrieval index, no mention of the service behind it, no owning team. The model cannot act on any of that, so it is noise competing with the sentences that matter.

Naming carries more weight than people expect

The name is the shortest and most repeated part of the description, and in a long tool list it is often what a decision turns on. Three rules cover most of it.

Verb plus noun, always. get_order_status, cancel_subscription, search_help_articles. A bare noun such as orders describes a resource, not an action, forcing the model to infer the verb — and it will occasionally infer a different one than you meant. The verb also encodes side effects: get_ and search_ read, create_ and cancel_ write, and that distinction does quiet safety work every time the model chooses.

No internal jargon or codenames. If a tool is called query_atlas because the service behind it is named Atlas, the model has no idea what Atlas is and must lean entirely on the description to recover. Names should be legible to anyone who has never seen your architecture diagram — the model, and every engineer joining next quarter.

search_v2 is a trap. To the model, search and search_v2 are near-identical candidates separated by a token that means nothing semantically, so it splits traffic between them arbitrarily. If v2 replaces v1, delete v1. If both must exist, name them after what they do — search_help_articles and search_product_catalogue — and the ambiguity disappears with the version number. Same for _new and _legacy: they encode migration history, not purpose.

Four ways to fix near-duplicates

This is the hardest case, and where teams waste the most time. Two tools overlap, the model splits traffic between them, and the reflex is to keep editing both descriptions until the confusion stops. Sometimes that works. Often the answer is structural — no amount of prose will fix a boundary that is in the wrong place. There are four legitimate responses.

Option two is underrated and should nearly always be tried first, because it is free and frequently resolves the problem outright. If search_help_articles and search_runbooks both open with "Search documentation for...", the first thing the model reads about each is identical. Rewrite them to open on the difference — "Search customer-facing published help articles..." against "Search internal engineering runbooks..." — and the discriminator arrives before the shared vocabulary rather than after it.

Option four is the one teams avoid, and it is often correct. A tool that never wins a case in your eval is not neutral — it is a permanent distractor competing for attention on every request, so deleting it improves every other tool.

What is not on this list is a system-prompt rule saying "prefer search_help_articles over search_runbooks". It works briefly, and puts the knowledge in the wrong place: the tools then behave correctly only inside this one agent, and the rule is invisible to whoever edits the tool next. Keep per-tool knowledge in the tool. Our guide to system prompt design for production AI agents covers where that boundary sits.

Parameter descriptions decide the arguments

A model that has chosen the right tool can still produce a schema-valid call that is wrong in a way no validator catches. The typing work — enums, required fields, bounds — belongs with the schema mechanics in the companion guide; what concerns us here is the natural-language text hanging off each parameter, where the model learns what a value should look like.

Four things earn their place there. Units, stated loudly, because a number is silent about its own scale and a weight field will receive kilograms sooner or later. Formats, with a real example, especially anything regional. Defaults, so the model knows what omission does. And what empty means — the question nobody documents and every model eventually gets wrong.

Regional formats are the clearest illustration, and they matter to anyone shipping into more than one market. A postal code field serving both the United Kingdom and India accepts two entirely different things: a UK postcode is alphanumeric with an internal space, an Indian PIN code is six digits with none. Given a bare "postal code" description, a model normalises the input to whatever pattern it saw most recently — and you get a support ticket from Coimbatore about deliveries that will not quote.

"properties": {
    "postal_code": {
        "type": "string",
        "description": (
            "Delivery postal code. For GB addresses, pass a full UK "
            "postcode including the internal space, e.g. 'EC2A 4NE'. "
            "For IN addresses, pass the 6-digit PIN code with no "
            "space, e.g. '600096'. Never pass a city or area name "
            "here - use the address_line fields for those."
        ),
    },
    "country": {
        "type": "string",
        "enum": ["GB", "IN"],
        "description": "ISO 3166-1 alpha-2 code of the delivery address.",
    },
    "weight_grams": {
        "type": "integer",
        "minimum": 1,
        "description": (
            "Parcel weight in GRAMS, not kilograms. 2 kg is 2000. "
            "Round up to the nearest gram."
        ),
    },
    "carrier": {
        "type": "string",
        "enum": ["royal_mail", "dpd_uk", "delhivery", "bluedart"],
        "description": (
            "Restrict the quote to one carrier. OMIT this field to "
            "quote every carrier serving the destination - omitting "
            "is not the same as passing an empty string, which is "
            "rejected."
        ),
    },
}

The carrier field is the one to study. Omission and emptiness are different states with different meanings, and a model with no guidance treats them as interchangeable. The same applies wherever null, empty string, empty list and absent could each plausibly mean "all", "none" or "unchanged" — say which, every time.

The broader principle — that a schema and its prose are one artefact the model reads together — is covered from the function-calling side in our explainer on reliable function schemas for agents. And when the descriptions are right and selection is still unreliable, the problem has moved to the model rather than the copy: our guide to fine-tuning for tool-call accuracy picks up where prompting stops.

The eval you are probably missing

Everything above is judgement until you measure it. Teams that have invested seriously in evaluating agent output — trajectory scoring, outcome checks, judge rubrics — often have no test for the decision that determines whether any of it is relevant. A tool-selection eval is the cheapest high-value suite you can build, and it takes an afternoon.

Building the case set

Collect realistic user requests and label each with the tool that should be called. Four rules make the set worth having. Harvest from real transcripts, because invented cases use your vocabulary and real users do not. Include a NO_CALL label for requests where the model should answer directly or ask a clarifying question — over-calling is a real failure mode, and an eval that cannot express it will never catch it. Weight the set towards boundaries, where the information is. And draw from both your markets: a request phrased by a user in Leeds and the same intent phrased in Hyderabad can route differently.

Five to ten cases per tool is enough to start. Selection failures are systematic rather than random, so an ambiguous description misfires on most cases that touch it, not on one in fifty.

A minimal harness

Keep the model call behind one small function so the suite outlives any provider decision. Everything else is counting.

import collections

# Each case: one realistic user request + the tool that should be called.
# Use "NO_CALL" when the agent should answer directly or ask a question.
CASES = [
    {"request": "why was I charged twice in July?", "expected": "search_help_articles"},
    {"request": "where has order 4821 got to?",     "expected": "get_order_status"},
    {"request": "refund the damaged speaker",        "expected": "refund_order"},
    {"request": "thanks, that's all",                "expected": "NO_CALL"},
    # ... 5-10 cases per tool, harvested from real transcripts
]


def predict_tool(request, tools):
    """Provider-agnostic: send ONE user turn plus the tool list, and return
    the name of the first tool the model calls, or None if it answered."""
    # wire your own provider client in here — the rest of the harness is provider-agnostic
    response = llm.call(messages=[{"role": "user", "content": request}], tools=tools)
    calls = [block for block in response.content if block.type == "tool_use"]
    return calls[0].name if calls else None


def run_selection_eval(cases, tools):
    matrix = collections.Counter()
    for case in cases:
        predicted = predict_tool(case["request"], tools) or "NO_CALL"
        matrix[(case["expected"], predicted)] += 1
    return matrix


def report(matrix, tool_names):
    labels = tool_names + ["NO_CALL"]
    print("expected \\ predicted".ljust(22), "".join(l[:9].rjust(11) for l in labels))
    for expected in labels:
        row = "".join(str(matrix[(expected, p)]).rjust(11) for p in labels)
        print(expected.ljust(22), row)
    print()
    for name in labels:
        hits = matrix[(name, name)]
        called = sum(matrix[(e, name)] for e in labels)      # column total
        should = sum(matrix[(name, p)] for p in labels)      # row total
        precision = hits / called if called else 0.0
        recall = hits / should if should else 0.0
        print(f"{name:22} precision={precision:.2f}  recall={recall:.2f}  n={should}")

Run it and you get a confusion matrix over tools: rows are what should have been called, columns what was called. That single table answers questions no amount of reading descriptions will.

Reading the matrix

The diagonal is the boring part. The information is everywhere else, and each pattern maps to a specific action.

Per-tool precision and recall make the same information easier to track over time. Low recall means a tool is missed when it should fire — a use-this-when problem. Low precision means it fires when it should not — a boundary problem. A tool with low precision and a hot column is your catch-all, whatever its name says.

Wire the suite into CI beside whatever end-to-end agent tests you already run. If you have built the trajectory and outcome evaluation described in our guide to evaluating AI agents by trajectory, tool call and outcome, this sits one level below it and runs far faster, because each case is a single model turn with no tool execution. That speed is what makes it usable as a gate on every change.

Keeping descriptions honest as the toolset grows

Descriptions drift. Behaviour changes, a filter is added, a data source is swapped, and the description keeps describing the version that shipped eighteen months ago. Unlike a stale code comment, a stale tool description is not merely misleading to a future reader — it is actively steering production behaviour on every request.

The structural point is the one most teams miss: every new tool is a change to every existing tool. Tools compete for selection. Adding an eleventh does not leave the other ten untouched; it introduces a candidate that may partially overlap several, and the decision boundary shifts accordingly. This is why a toolset that worked well at six tools degrades at fifteen without anyone touching the original descriptions.

A short review checklist when you add tool N+1, run before merge:

1. Read it in the list. Print all N+1 names and descriptions together, nothing else, and look for the overlap you did not intend.
2. Name check. Verb plus noun, no codenames, no version suffixes, no collision with an existing pair.
3. Boundary check. Does the new description name its nearest neighbour in a do-not-use-this-when clause? Does the neighbour name it back?
4. Add cases. Five to ten labelled requests, plus at least two boundary cases against that neighbour.
5. Re-run everything. The full matrix, not the new rows, compared against the last committed run.
6. Justify the count. If the toolset has grown past what you can hold in your head, that is the signal, not the token cost.

That last point has a natural limit. Past a certain number of tools — as of August 2026, in the agents I have worked on that threshold has arrived in the low dozens rather than the low hundreds — the answer stops being better descriptions and becomes not showing the model every tool at once. Retrieving a relevant subset per request, or deferring schemas until needed, changes the problem shape entirely; our guide to dynamic tool retrieval when your agent has 200 tools covers that architecture, and this guide does not duplicate it. Retrieval does not make description quality less important, though. It makes it more important, because the retrieval step matches against those same descriptions, and a vague one now fails twice.

Key takeaways

Tool descriptions are the highest-leverage untested prompt in most agents. The fix is not a bigger model or a longer system prompt; it is a few dozen words per tool, and a small eval that tells you whether they work.

• Classify before you fix. Selection and execution failures look identical in a bug report and have different cures.
• Write for the model. Capability line, use-this-when, do-not-use-this-when with a named redirect, coverage limits, one worked example.
• Name with a verb and a noun. No codenames, no _v2, no migration history in the name.
• Near-duplicates have four fixes — cross-reference, lead with the discriminator, merge behind an enum, or delete.
• Parameter descriptions carry units, formats, defaults and what empty means. A UK postcode and an Indian PIN code are not the same string.
• Build the confusion matrix. Rows are what should have been called; the off-diagonal cells say what to rewrite, merge or delete.
• Re-run it on every new tool, model change and provider change, because adding a tool changes all the others.