D5
Multi-Tenant Isolation, Cache Safety, and Contamination
- 01Construct tenant-safe cache keys that encode every output and authZ dimension
- 02Explain why a semantic cache can disclose one tenant's answer to another
- 03Identify cross-user contamination from shared mutable state and pool/concurrency bugs
- 04Apply per-request isolation and full-identity namespacing to memory stores
Shared system, many tenants
topic 21Multi-tenant isolation is your job, not the provider’s. One missing tenant_id filter leaks data across tenants — worst on vector queries, where a forgotten filter returns another tenant’s nearest neighbors. Use namespaces, metadata filters, or separate indexes, and treat the filter as non-optional.
Cache safety
A semantic cache matches on meaning, not exact text. So Tenant B’s near-duplicate question can hit Tenant A’s cached private answer unless the key includes identity. Every cache key must encode all output and authZ dimensions — tenant, user, permission set, model, prompt version, locale — and invalidate when permissions change.
Common MisconceptionProvider prompt caching will leak my data to other customers.
CorrectionProvider caches are scoped to your org; the real risk is YOUR semantic cache leaking across YOUR tenants.
On an embedding plane, Tenant B's similar query drifts into Tenant A's cached similarity radius and returns A's private answer — until a 'tenant_id in key' toggle walls the plane into per-tenant regions and B misses safely.
Text description
On an embedding plane, Tenant B's similar query drifts into Tenant A's cached similarity radius and returns A's private answer — until a 'tenant_id in key' toggle walls the plane into per-tenant regions and B misses safely.
Cross-user contamination
The scariest leaks aren’t model bugs — they’re concurrency and state bugs. A module-level or global buffer in an async server, a singleton session object, or a mis-keyed memory store bleeds one user’s tokens into another’s response. The March 2023 ChatGPT incident — a redis-py async client bug — exposed some users’ data to other users (verify the exact exposed fields before quoting).
Production failure modes
- A missing tenant filter on a vector query returns another tenant's neighbors.
- A semantic cache serves Tenant A's answer to Tenant B via a near-duplicate query.
- A module-level/singleton buffer interleaves concurrent users' output.
- A mis-keyed long-term memory store injects another user's history.
- Permission revocation doesn't invalidate cached or remembered answers.
Retrieval Practice
Check one idea at a time
A multi-tenant assistant with a semantic cache returns Tenant A's confidential answer to Tenant B. Root cause?