RAG Architectures: Choosing a Retrieval Pattern for the Task

Reviewed and updated October 8, 2026.

Choose a retrieval pattern by the questions it must answer and the failures you can demonstrate. Begin with an approved document set, a simple search baseline and a small collection of representative questions. Adding a graph or another model before testing that baseline makes it harder to identify what improved.

Retrieval-augmented generation supplies retrieved material to a generator. The original RAG paper combines a parametric model with a non-parametric document index. That mechanism does not prescribe one complete enterprise architecture or establish that every additional component helps.

Match the pattern to the missing evidence

• Keyword retrieval: use an ordinary search index as a baseline for names, identifiers and exact phrases. Inspect whether the correct source appears before blaming generation.
• Vector retrieval: compare semantic matching when users describe an idea differently from the document. Keep document identifiers, versions and access metadata alongside each chunk.
• Hybrid retrieval: combine lexical and vector candidate sets. Azure AI Search's documented implementation runs both searches and merges rankings with reciprocal rank fusion. This is one implementation to evaluate, not a universal requirement.
• Reranking: score a bounded candidate set against the question before selecting context. It adds an inference step; test whether relevant evidence rises enough to justify its latency and operating cost.
• Entity or graph retrieval: consider explicit relationships when the question crosses people, products or agreements. Microsoft's GraphRAG documentation distinguishes entity-focused local search from global search over community reports. Global search is resource intensive.
• Structured lookup: query an authorized database or business API for exact current values. A generated explanation can accompany the returned records, but prose similarity should not determine an account balance.

An illustrative policy comparison

Suppose a service team asks, “Which warranty covers product AX-17 sold under contract C-42?” This example is a proposed evaluation, not a measured ALLTIPLY result. The correct evidence requires the product identifier, the applicable contract revision and the relevant warranty clause.

Run the same question against keyword search, vector search and a hybrid candidate set. Label the expected documents before inspecting answers. If vector search returns a similar product's warranty, add the exact identifier constraint or lexical candidate path. If the correct contract is retrieved but the answer uses a superseded clause, fix version selection. A graph is a candidate only if linking contract, product and amendment resolves a demonstrated relationship failure.

For each trial, record whether the expected evidence was retrieved, whether the answer's claims follow from it, whether citations identify the correct version and whether unauthorized material was excluded. Also record end-to-end time and the work needed to update the index. An answer that sounds right while citing the wrong contract fails this task.

Know when to stop adding architecture

A short, stable handbook may work with direct document context. A source search interface may be sufficient when users mainly need to locate documents. Neither requires an agent. Conversely, multi-step retrieval adds failure points when a request needs several dependent lookups. Put a limit on expansion, define what happens when evidence is missing and keep a visible route to a human owner.

Permissions, deletion propagation and freshness apply to every pattern. They cannot be repaired by a more fluent generator. Keep the simplest design that meets the agreed cases, and retain failed cases when comparing a new component.

For document questions tied to real decisions, see enterprise knowledge assistants. Request a project review with sample questions, source systems and the evidence an acceptable answer must show.