ALLMAcademy

S3

The Model Plans, Your Code Executes: Tool Calls & Idempotency

15 min2 optional deep dives

Treat the model as a planner, validate every tool request, and make side effects safe to replay.

  1. 01Describe the function-calling loop and explain why the model is a planner, not an executor
  2. 02Enumerate function-calling failure modes (hallucinated args, accuracy decay past ~10-20 tools, missing/over-eager calls, parallel races) and their mitigations
  3. 03Treat every generated tool argument as untrusted input requiring schema shape, business rules, and injection defense
  4. 04Apply idempotency so an at-least-once loop produces exactly-once side effects
Narration — The Model Plans, Your Code Executes: Tool Calls & Idempotency
0:00 / 0:00

The function-calling loop

topic 9

You declare tools (name, description, JSON-schema params). The model emits a call — and runs nothing. Your code executes it and returns the result; the loop continues.

Levers that buy reliability:

  • tool_choice — auto / required / none / forced. Forcing one tool is the canonical reliable-extraction trick.
  • Tool count — accuracy decays past ~10–20 overlapping tools (a model-dependent heuristic); buy it back with tool retrieval, namespacing, or sub-agents.
  • Parallelism — disable parallel calls for dependent operations to avoid races (parallel_tool_calls).

Tool contracts and arguments as untrusted input

Tool descriptions are read by the model — they are prompt engineering, so use clear names, examples, enums, and typed/required fields. But the arguments the model returns must be re-validated server-side as if a hostile user typed them.

Common MisconceptionIf the arguments validate against the schema, it's safe to execute.

CorrectionSchema shape is not enough — you still need business rules, authorization, and injection defense (SQLi, command, path traversal, SSRF).

Common MisconceptionMore tools make the agent more capable.

CorrectionSelection accuracy decays as overlapping tools pile up; scope or retrieve tools instead.

Idempotency: exactly-once effects on at-least-once delivery

Agent loops are at-least-once: timeouts, retries, and re-issued calls duplicate side effects. Make writes idempotent (client idempotency keys deduped server-side; PUT/DELETE/upserts; dedup windows), split read vs write tools, and add timeouts / bounded retries / circuit breakers.

At-least-once vs exactly-onceInteractive · 2D

A duplicate tool call replays through the loop: with the idempotency key OFF the side-effect counter doubles (red); with the key ON the server hits a dedup cache and the counter is unchanged (green).

Loading diagram…
Text description

A duplicate tool call replays through the loop: with the idempotency key OFF the side-effect counter doubles (red); with the key ON the server hits a dedup cache and the counter is unchanged (green).

Production failure modes

  • Duplicate side effects from non-idempotent writes plus retries (double charges, double sends).
  • Hallucinated or missing arguments passed straight to execution.
  • Extraction breaking under tool_choice='auto' when a call was required.
  • Injection through tool args; parallel calls racing dependent operations.
References & deeper reading (2)

Retrieval Practice

Check one idea at a time

Question 1 of 2

Why must write tools be idempotent in an agent loop?