Tool use & function calling
The model emits a structured call (tool name + JSON arguments); your application validates and runs it, then returns the result to the model.
Tool use (also called function calling) lets an Large Language Model (LLM) ask for things it can't do alone: look up an order, query a database, run a calculation. You describe each tool with a name, a description and the inputs it takes.
The model never runs anything itself. It writes a structured request ("call get_order_status with order_id 48213"), your program decides whether to run it, runs it, and sends the answer back so the model can continue. This round trip is the building block of every AI agent.
A manager who can't leave the office fills in a request slip; an assistant checks it's allowed, fetches the file, and puts it on the desk.
Trusting arguments because they came from your own model. A customer_id parameter the model fills from conversation is an authorisation hole; identity must come from the session.
You send tool definitions (name, description, JSON Schema for input_schema/parameters) with the messages. A model trained for the format replies with one or more tool-call blocks (tool_use on Anthropic, function_call items on OpenAI's Responses API), each with an id and JSON arguments; several calls can come back in one response for parallel execution. Strict schema modes (e.g. OpenAI strict: true) use Structured output / decoding so arguments always parse. The host executes and replies with a tool-result message keyed to the call id; the model continues or answers.
The host is the trust boundary. Arguments are model output, so validate them, derive identity and tenancy from the session (never from an argument), enforce Least privilege, set timeouts, attach Idempotency keys to writes, and treat results as untrusted text that may carry Prompt injection (indirect). Tool definitions are prompt tokens on every request (a typical definition is 100–500 tokens) and their wording drives selection accuracy — see Tool design (agent–computer interface).
Native tool-use API with strict/structured arguments where offered; 3–10 tools; JSON Schema with enums and required fields; identity and tenant injected server-side; 10 s default tool timeout; idempotency key on every write; tool results capped at a few thousand tokens with a truncation hint; parallel tool calls on for independent reads.
E-commerce catalog assistant. Tool get_order_status(order_id: string). User: "Where is order 48213?" Model returns {"name":"get_order_status","input":{"order_id":"48213"}}. The host checks that order 48213 belongs to the signed-in customer, calls the orders API (120 ms), and returns {"status":"shipped","eta":"2026-10-15"} as a tool_result. Model: "Order 48213 has shipped; expected 15 October." Two model calls, ~1.4 s end to end.
Walk the round trip and point at the trust boundary: "the model proposes a call; my code decides whether it happens and with whose identity." Recommend strict schemas plus server-side authorisation for any tool that reads tenant data.
Expect the follow-up "what if the tool returns text that says 'ignore previous instructions'?" — answer with treating results as data, capability limits on later tools, and approval gates for writes.
Numbers worth knowing
- Tokens per tool definition
- 100–500name + description + schema; measure with the provider's token counter
- Model calls per single-tool answer
- 2one to emit the call, one to use the result
- Hidden tool-use system prompt
- a few hundred tokens per requestadded by providers when tools are present; see pricing docs
How to decide
| When | Do this |
|---|---|
| The model needs data or an action outside its weights | Expose a tool; don't paste the whole dataset into the prompt. |
| Arguments must always parse (downstream code depends on them) | Use strict schema mode or Structured output / decoding, plus server-side validation. |
| A tool writes, spends or sends | Idempotency key, least-privilege credential, and Human in the loop above a risk threshold. |
Cost maths
Each tool hop costs one extra model call that re-reads the context: cost ≈ (prefix + tools + history + result) × p_in + out × p_out. With 8 tools × 300 tokens = 2.4k tokens of definitions on every call, at $3 / M input (illustrative, Oct 2026) that is $0.0072 per call; at 2M calls/month ≈ $14.4k/month for definitions alone, or ≈ $1.4k with Prompt caching reads at 0.1×.
Operating it
- Tool definitions exceed ~10% of the input tokens per call
- Argument validation failures above 2% of calls
- More than one tool returns results larger than 5k tokens
- Argument schema-validation failure rate (alert > 2%)
- Tool error and timeout rate per tool (alert > 5%)
- Calls where argument identity ≠ session identity (alert on any)
- p95 tool latency per tool (alert above its own SLO)
When it goes wrong
A support assistant occasionally showed one customer another customer's order history.
A customer complaint, confirmed by audit logs where customer_id in the tool arguments differed from the session's customer.
get_orders(customer_id) trusted the model-supplied id; a user typing someone else's email led the model to pass that customer's id.
Removed customer_id from the schema and bound it from the authenticated session; added a test that any tool touching tenant data ignores identity arguments.
Related terms
Where it shows up
Sources
- Tool use with Claude — Anthropic, 2026docs
- Function calling — OpenAI, 2025docs
- Toolformer: Language Models Can Teach Themselves to Use Tools — Schick et al., 2023paper
Check yourself
Three questions. Every option is explained once you answer.