ALLMAcademy

S2

Designing a Multi-Turn Conversation

12 min2 optional deep dives

Decide what each turn knows, what carries forward, and when a conversation should reset.

  1. 01Separate persistent instructions from user and assistant turns
  2. 02Choose what history to retain, summarize, or discard
  3. 03Define reset and escalation behavior before the context becomes unreliable

A conversation is repeated request assembly

mental model

The model does not “remember” a chat because a window stays open. Your application builds each new request from:

  • stable system rules,
  • selected earlier turns,
  • summaries or retrieved facts,
  • tool results,
  • and the current user message.

Define what carries forward

Use a simple policy instead of “send everything”:

Information Typical treatment
Safety and product rules Keep stable and first
Current goal and constraints Keep until the goal changes
Recent turns Keep while they affect the next answer
Old details Summarize or retrieve when needed
Unrelated topic Reset or isolate in another thread

Example: support conversation

The user says, “My invoice is wrong.” Later they say, “The duplicate charge is from July.”

Carry forward the invoice identifier and the fact that the issue is a duplicate charge. Do not keep unrelated small talk or old troubleshooting steps after the issue changes.

Common MisconceptionA long conversation should always include every earlier message.

CorrectionEvery retained turn spends tokens and can distract the model. Keep only what still changes the next decision.

Deep dive: System messages and role boundaries

System rules are not a security boundary by themselves, but they are the most stable instruction layer. Keep them separate from user content, and never paste untrusted documents into the same instruction channel.

References & deeper reading (2)

Retrieval Practice

Check one idea at a time

Question 1 of 3

A user changes from planning a trip to debugging code. What is the safest default?