Types of RAG Architectures

This README summarizes several common Retrieval-Augmented Generation (RAG) architecture patterns, starting from the basic setup and moving toward more advanced variants.

Table of contents

• Basic RAG Architecture
• Types of RAG Architectures
  • 1. Simple/Basic RAG
  • 2. Simple RAG with Memory
  • 3. HyDe (Hypothetical Document Embedding)
  • 4. Speculative RAG
  • 5. Agentic RAG
  • 6. Graph RAG
• References

---

Basic RAG Architecture

Before moving to complex variants, it helps to understand the basic RAG pattern.

Basic RAG uses a dual-component (Retriever + Generator) strategy to improve text generation by grounding responses in retrieved, external data. The retriever pulls relevant documents, and the generator uses those documents as context to produce answers that are more accurate and contextually relevant.

Diagram (placeholder)

In the referenced GIF, parts 1–5 correspond to retrieval, and parts 6–7 correspond to generation.

---

Types of RAG Architectures

There are many ways to modify RAG. Below are several widely used variants.

1. Simple/Basic RAG

This is the baseline RAG pattern described above: retrieve relevant documents, then generate an answer using that retrieved context.

---

2. Simple RAG with Memory

Simple RAG can be improved by adding a memory (storage) component so the system can retain information from previous interactions. This helps with multi-turn conversations and tasks that require context continuity across multiple queries.

Workflow

• Query Input: The user submits a query or prompt.
• Memory Access: The model retrieves past interactions or data stored in its memory.
• Document Retrieval: It searches the external database for new relevant information.
• Generation: The model generates a response by combining retrieved documents with the stored memory.

---

3. HyDe (Hypothetical Document Embedding)

HyDe generates a hypothetical document based on the query first. Instead of directly retrieving from the database, it embeds what an “ideal” document might look like for the query, then uses that representation to guide retrieval—often improving relevance and result quality.

Workflow

• Query Input: The user provides a prompt or question.
• Hypothetical Document Creation: The model generates an embedded representation of an ideal response.
• Document Retrieval: Using the hypothetical document, the model retrieves actual documents from a knowledge base.
• Generation: The model generates an output based on the retrieved documents, influenced by the hypothetical document.

---

4. Speculative RAG

Speculative RAG encourages the model to make educated guesses when retrieved data is insufficient or ambiguous. It is designed for situations where complete information may not be available, but the system still needs to provide a useful response. The model generates plausible conclusions using both retrieved patterns and the language model’s broader knowledge.

Workflow

1. User Query: The user begins entering a query.
2. Contextual Analysis: The system analyzes the user’s behavior and context.
3. Predictive Data Retrieval: Relevant data is pre-retrieved based on predicted needs.
4. Speculative Response Generation: Generates a tentative response.
5. User Feedback Collection: User interacts with the generated response.
6. Refined Response Generation: Response is updated based on feedback.
7. Final Output to User: The user receives a quick, relevant answer.

---

5. Agentic RAG

Agentic RAG makes retrieval + generation more autonomous by introducing agent-like behavior. The system can perform multi-step tasks, proactively interacting with multiple data sources or APIs. A key idea here is using Document Agents (one per document) orchestrated by a Meta-Agent that chooses strategies and coordinates retrieval based on query complexity.

Workflow

• Query Input: The user submits a complex query or task.
• Agent Activation: Multiple agents are activated. Each Document Agent is responsible for a specific document and can answer questions or summarize it.
• Multi-step Retrieval: The Meta-Agent coordinates Document Agents and ensures the most relevant information is retrieved.
• Synthesis and Generation: The Meta-Agent integrates agent outputs into a comprehensive, coherent response.

---

6. Graph RAG

Graph RAG uses graph-based structures (e.g., knowledge graphs) to retrieve and organize information based on entity relationships. This is especially useful when understanding the connections between concepts matters (legal precedent networks, semantic web, social networks, etc.). It retrieves both facts and the connections among them.

Workflow

• Query Input: The user submits a complex query requiring understanding of relationships and contextual connections.
• Agent Activation: A Graph Agent maps and traverses the graph structure to identify key entities and relationships.
• Multi-step Retrieval: The Graph Agent traverses direct and indirect connections to retrieve contextually rich information.
• Synthesis and Generation: The Graph Agent integrates information from multiple graph segments while preserving semantic relationships.

References

• https://mlubbad.medium.com/top-6-different-rag-architectures-2efed3f8a868
• https://www.bluetickconsultants.com/blogs/from-rag-to-graphrag-transforming-information-retrieval-with-knowledge-graphs.html
• https://medium.com/@zbabar/design-variations-of-the-rag-architecture-9ff5d1d5b4de
• https://medium.com/@rupeshit/mastering-the-25-types-of-rag-architectures-when-and-how-to-use-each-one-2ca0e4b944d7