ALLMAcademy

Primer

Retrieval, State, and Memory

13 min2 optional deep dives

Separate what a call can see now from what an application searches, tracks, checkpoints, resumes, or remembers.

  1. 01Distinguish current-call context, retrieval, RAG, thread state, checkpoints, durable tasks, and long-term memory
  2. 02Compare lexical, semantic, and GraphRAG retrieval at an orientation level
  3. 03Choose the right scope and lifetime for information an application must retain

Name the storage and execution layer

orientation
Term What it does Typical scope
Current-call context Gives one model invocation information it can directly use One call
Retrieval Searches stored information and selects candidates One search
RAG Retrieves evidence and places selected evidence into a model request One answering step
Thread state Tracks messages, variables, and decisions for one ongoing thread One conversation or run
Checkpoint Saves a workflow snapshot so execution can resume One workflow
Durable task Represents long-running work with an ID, status, and result One operation
Long-term memory Retains useful information for future threads Cross-thread
A stateless call inside a stateful systemInteractive · 2D

The application assembles current context from thread state, retrieved evidence, and selected memory, then checkpoints workflow progress and tracks durable work outside the model call.

Loading diagram…
Text description

The application assembles current context from thread state, retrieved evidence, and selected memory, then checkpoints workflow progress and tracks durable work outside the model call.

Retrieval is a selection step

Lexical retrieval matches words, phrases, and exact tokens. It is strong for names, product codes, error strings, and quoted text. Semantic retrieval embeds queries and documents as vectors so differently worded passages can match by meaning. It is strong for paraphrases and conceptual similarity, but may underweight an exact identifier.

RAG—retrieval-augmented generation—uses retrieval to select evidence, then adds that evidence to the current model request. The model cannot use a retrieved document until some layer places the relevant content or a usable representation into its context.

State is not one bucket

Thread state might hold the latest user goal and completed tool results. A checkpoint records enough workflow state to restart after a pause or crash. A durable task gives long-running work a stable handle so a client can inspect status, cancel, or fetch a later result.

Long-term memory crosses thread boundaries. It needs a reason to exist, an owner, a retention rule, a correction path, and a retrieval policy. Storing everything makes future selection harder and creates privacy and staleness risks.

Common MisconceptionRAG gives the model permanent knowledge.

CorrectionRAG selects evidence for a request. Unless the application stores something separately, that evidence disappears from the model's view after the call.

Deep dive: Where GraphRAG fits

GraphRAG indexes a corpus into entities, relationships, and community structures or summaries. That can help with questions about connections among people, organizations, events, or themes across a large corpus—not just the nearest passage to one query.

The tradeoff is additional extraction, indexing, storage, and query work. Use it for relationship-heavy or corpus-level questions that simpler lexical, vector, or hybrid retrieval does not answer well. The dedicated retrieval modules cover design and evaluation in depth.

Category mistakes

  • Calling conversation history memory without defining who stores it or for how long.
  • Treating a checkpoint as protection against duplicate external side effects.
  • Using semantic search alone for exact IDs, names, or error codes.
  • Building a knowledge graph before proving simpler retrieval cannot answer the target questions.
References & deeper reading (3)

Retrieval Practice

Check one idea at a time

Question 1 of 3

Match each mechanism to its primary role.

Match each item to its pair.