Memory and RAG solve different context problems

A language model handles the context supplied with the current request. Memory and RAG are two ways to assemble that context, but they should not be merged into one vague “agent brain.”

Memory normally carries selected messages, summaries or task state across turns. RAG searches an external collection and supplies the chunks most relevant to the current question. A third category—live tools and systems of record—retrieves transactional facts such as account status, stock, bookings or permissions.

How the three context layers work together

Text equivalent: the workflow verifies the requester and session, selects relevant conversational state, retrieves approved documents and queries live systems only when needed. The model receives each source with a clear label. Validation and approval happen before the final response or action.

When should you use agent memory?

Use memory when continuity improves the interaction: remembering which product the user is discussing, which clarification was already answered, or which stage of a multi-step task has been completed. Keep the stored state as small and explicit as possible.

• Short-term window: a limited number of recent messages. Simple, but old details fall out and long conversations become expensive.
• Conversation summary: a compact representation of earlier turns. Efficient, but a mistaken summary can persist.
• Structured task state: named fields such as selected location, confirmed date or missing data. Usually safer than asking the model to rediscover state from prose.
• Longer-lived preferences: only when the purpose, consent or other basis, retention and user controls are clear.

n8n documents Simple Memory for a configurable chat-history window and separate memory services for persistence. The storage choice does not remove the need for correct session keys, isolation and retention.

When should you use RAG?

Use RAG when an answer depends on approved external content that is too large, specialised or frequently updated to place in every prompt. The ingestion workflow fetches documents, splits them into chunks, creates embeddings and stores them with metadata. The query workflow retrieves a limited set of relevant chunks.

Good metadata is often as important as semantic similarity. Filter by product, locale, audience, policy version, effective date and access group before asking the model to answer. A highly similar but expired policy is still the wrong source.

RAG is not a guarantee of truth

Retrieval can miss the best document, return an irrelevant chunk or surface two conflicting versions. The answer can also go beyond the supplied evidence. Test retrieval relevance and answer groundedness separately, and provide a “no reliable source found” route.

What should stay outside both memory and RAG?

• Current account balance, order state, inventory and booking availability.
• Identity, authentication and authorisation decisions.
• Permission to send, delete, purchase, refund or change a record.
• Secrets and unrestricted credentials.
• Legal, clinical or financial conclusions that require authorised review.

Fetch changing transactional facts from the system of record with an exact tool call. Use memory to preserve task continuity and RAG to explain the relevant policy; do not use either as the source of truth for a current transaction.

Choose the right design

Failure paths and tests

Success criteria

• Every request uses the correct authenticated session and tenant boundary.
• Structured task state replaces unnecessary transcript replay.
• Retrieved chunks include document ID, version and access metadata.
• The system can say that no reliable source was found.
• Transactional truth comes from exact live tools.
• Retention and deletion behaviour are tested for both memory and indexed knowledge.