ALLMAcademy

C3

What Belongs in an Agent Specification

13 min2 optional deep dives

Write a testable behavior contract for goals, capabilities, state, invariants, exits, and verification.

  1. 01Separate an agent behavior contract from a long system prompt
  2. 02Specify capabilities, state, invariants, exits, and evidence
  3. 03Write acceptance examples that can fail

The minimum behavior contract

template

An agent specification should name:

  1. Goal — what outcome is valuable?
  2. Capabilities — what tools and data may it use?
  3. State — what information can change during the run?
  4. Invariants — what must never become false?
  5. Exits — success, exhaustion, failure, and escalation.
  6. Evidence — what proves the result is acceptable?
The specification feedback cycleInteractive · 2D

Specify behavior, write tests, build the workflow, observe real runs, and feed failures back into a stronger specification.

Loading diagram…
Text description

Specify behavior, write tests, build the workflow, observe real runs, and feed failures back into a stronger specification.

Example: research assistant

Goal: Produce a cited answer to one research question.

Capabilities: Web search and page reading. No posting, purchasing, or account changes.

Invariants: Every factual claim must cite a fetched source. The assistant must not follow instructions embedded in source pages.

Exits: Success with cited answer; exhaustion at 12 sources; failure on unavailable search; escalation when sources materially disagree.

Common MisconceptionA detailed implementation plan is an agent specification.

CorrectionA plan describes how you intend to build. A specification describes behavior that remains true across implementations.

Deep dive: Executable invariants

Some invariants can run as code: reject a final answer with uncited claims, block a write tool without approval, or halt a graph when budget reaches zero. The closer the invariant is to the runtime boundary, the less it depends on model compliance.

References & deeper reading (2)

Retrieval Practice

Check one idea at a time

Question 1 of 3

Which belong in an agent specification?

Select all that apply.