From Specification to Graph and Tests

Map contract language to runtime controls

worked example

Specification	Runtime artifact
“Stop after 8 attempts”	Budget guard and exhausted terminal state
“Every claim has a source”	Citation validator before success
“Human approves payment”	Interrupt before the payment tool
“Resume after failure”	Checkpointer and idempotent tools

Backpropagate bugs into the spec

Suppose a crash after send_email causes the graph to retry and send twice.

Do not stop at “fix the retry.” Add the missing invariant:

Every side-effecting tool call uses a stable idempotency key, and replay cannot create a second effect.

Then add a regression test that simulates the crash window.

Production failure modes

• The specification is updated to match broken code instead of fixing the implementation.
• An invariant exists only in prose and has no runtime enforcement.
• Tests cover the happy path but not exhaustion, failure, or replay.
• The graph adds a new capability without updating permissions and threat assumptions.

Deep dive: Trace-based acceptance tests

A trace records node order, inputs, outputs, tool calls, budgets, and terminal state. Save a redacted trace from a real failure and turn it into an acceptance scenario. This tests the system’s behavior, not just one function.