Unit 4: Retrieval-Augmented Generation and Multi-Agent Collaboration

I. Orientation — Grounding and Cooperative Intelligence

Retrieval-Augmented Generation (RAG) combines information retrieval with a generative model so that responses can be based on external evidence rather than only on model parameters. Multi-agent systems divide reasoning or action among autonomous agents that communicate, coordinate, and allocate tasks toward individual or shared objectives.

• Defining properties:
  • External grounding: RAG supplies retrieved documents, database records, or tool outputs as context for generation.
  • Semantic representation: Embeddings encode meaning as numerical vectors, enabling similarity-based retrieval.
  • Modular pipeline: Ingestion, chunking, indexing, retrieval, reranking, generation, and evaluation can be improved independently.
  • Agency: An agent observes its environment, selects actions, and receives resulting observations or feedback.
  • Collaboration: Multiple agents exchange information, divide work, resolve dependencies, and combine results.
  • Core assumption: Better evidence and coordination generally improve reliability, but neither guarantees correctness; retrieval errors and agent failures can propagate.

II. Embeddings and Semantic Search — Representing Meaning as Vectors

A. Embeddings and semantic search

Embeddings map text or other data into a vector space where semantically related items are expected to lie near one another.

• Embedding function: A model computes (f(x)=\mathbf{e}), where (x) is an input such as a sentence and (\mathbf{e}\in\mathbb{R}^{d}) is a vector with (d) numerical dimensions.
• Dense representation: Unlike one-hot vectors, dense embeddings distribute information across dimensions; a 768-dimensional vector contains 768 floating-point values rather than one coordinate per vocabulary item.
• Semantic retrieval: The query and documents are embedded using compatible models, and documents nearest to the query vector are returned.
• Cosine similarity: Directional similarity is commonly measured by:

cos(q, d) = (q · d) / (||q||₂ ||d||₂)

• (q) is the query embedding, (d) is a document embedding, (q\cdot d) is their dot product, and (||\cdot||_2) is Euclidean norm.
• Scores approach (1) when vectors point in similar directions.
  • Concrete example: “How do I reset my password?” may retrieve “Steps for recovering account access” despite sharing few exact words.
  • Contrast with lexical search:
    1. Lexical search: Methods such as BM25 reward matching terms and term rarity, making them effective for names, identifiers, and exact phrases.
    2. Semantic search: Dense similarity captures paraphrases and conceptual relationships but may overlook exact rare tokens.
  • Limitations: Domain mismatch, multilingual variation, outdated embedding models, and ambiguous queries can produce misleading neighbors.

III. Chunking and Indexing — Preparing Knowledge for Retrieval

A. Chunking and indexing strategies

Chunking determines the units retrieved, while indexing organizes those units so relevant evidence can be found efficiently.

• Fixed-size chunking: Text is divided by token or character count, such as 400-token chunks with a 50-token overlap; implementation is simple, but boundaries may split arguments or definitions.
• Structure-aware chunking: Headings, paragraphs, tables, functions, or document sections define boundaries; a section titled “Refund Conditions” remains a coherent retrieval unit.
• Recursive chunking: A system first splits on large boundaries such as sections, then paragraphs and sentences until each chunk satisfies a size limit.
• Semantic chunking: Adjacent sentences are grouped while their embedding similarity remains high; a sharp topic change starts a new chunk.
• Overlap trade-off: Overlap preserves context across boundaries but increases storage and can return redundant evidence. If 400-token chunks overlap by 100 tokens, each new chunk advances 300 tokens.
• Metadata: Each chunk should retain fields such as document_id, title, section, timestamp, access level, and source location for filtering and citation.
• Indexing alternatives:
  1. Sparse index: An inverted index maps terms to documents and supports lexical scoring.
  2. Dense index: A vector index stores embeddings for nearest-neighbor search.
  3. Hybrid index: Sparse and dense results are fused, combining exact-token precision with semantic recall.
• Design principle: Small chunks increase retrieval precision but may lack context; large chunks preserve context but may dilute the relevant passage.
• Operational requirement: Changed documents must be re-chunked and re-embedded, while deleted or access-restricted chunks must be removed or filtered.

IV. Vector Databases — Storing and Searching Embeddings

A. Vector databases

A vector database stores embeddings with identifiers and metadata and performs efficient nearest-neighbor search over large collections.

• Stored record: A typical record contains id, embedding vector, source text or pointer, and metadata such as {department: "legal", year: 2026}.
• Similarity metrics: Common choices are cosine similarity, dot product, and Euclidean distance; the selected metric should match the embedding model’s training assumptions.
• Exact search: Comparing a query against all (N) vectors gives accurate results but has approximately (O(Nd)) work for (d)-dimensional vectors.
• Approximate nearest-neighbor search: Structures such as Hierarchical Navigable Small World graphs or inverted-file indexes reduce latency by searching promising regions rather than every vector.
• Filtering: Metadata predicates can restrict retrieval, for example to documents where access_level = "public" and year >= 2025.
• Database responsibilities: Index construction, top-(k) retrieval, persistence, replication, updates, deletion, and namespace or tenant isolation.
• Trade-offs: Higher recall often requires more memory or search time; aggressive approximation improves latency but may miss the true nearest vectors.
• Security constraint: Authorization should be enforced before evidence reaches the model because generation-time instructions cannot reliably protect restricted content.

V. Retrieval Pipelines — Evidence for Knowledge-Grounded Reasoning

A. Retrieval pipelines for knowledge-grounded reasoning

A retrieval pipeline converts a user request into selected evidence and supplies that evidence to a model under explicit grounding instructions.

• Offline ingestion: Documents are cleaned, normalized, chunked, embedded, enriched with metadata, and inserted into searchable indexes.
• Query processing: The system may classify intent, expand abbreviations, rewrite conversational queries, or decompose a multi-part request.
• Candidate retrieval: Dense, sparse, or hybrid search returns a broad candidate set; for example, retrieve 30 chunks before selecting the best 5.
• Reranking: A cross-encoder or language model jointly examines the query and each candidate to improve relevance ordering.
• Context construction: Selected chunks are deduplicated, ordered, labelled with source identifiers, and fitted within the model’s context window.
• Generation constraint: The prompt should require answers from supplied evidence, citations to chunk identifiers, and an explicit statement when evidence is insufficient.
• Basic pipeline:

q'         = rewrite(query)
candidates = hybrid_retrieve(q', top_k=30)
evidence   = rerank(q', candidates)[:5]
answer     = generate(query, evidence)

• query is the user input, (q') is its retrieval-oriented rewrite, candidates are initial matches, and evidence is the final grounded context.
  • Iterative retrieval: An agent may retrieve, identify a missing fact, formulate a follow-up query, and retrieve again; this supports multi-hop questions whose facts occur in separate documents.
  • Evaluation:
• Retrieval recall: Whether relevant evidence appears among the top-(k) results.
• Faithfulness: Whether answer claims are supported by retrieved evidence.
• Answer relevance: Whether the response addresses the original request.
• Latency and cost: Time and computational expense across embedding, search, reranking, and generation.
  • Failure modes: Weak queries, poor chunks, stale indexes, irrelevant reranking, prompt injection inside retrieved text, and unsupported model synthesis can all break grounding.

VI. Multi-Agent Systems — Distributed Autonomous Problem Solving

A. Foundations of multi-agent systems

A multi-agent system contains multiple autonomous entities whose actions and interactions determine system-level behavior in a shared or connected environment.

• Agent model: An agent follows an observation–decision–action loop:

observation = perceive(environment)
action      = policy(observation, memory, goal)
result      = act(action)

• policy is the decision rule, memory stores relevant state, and goal defines the desired outcome.
  • Autonomy: Agents select actions without requiring a human decision at every step, although policies and permissions constrain them.
  • Specialization: Agents may have distinct roles, such as planner, retriever, coder, verifier, or database operator.
  • Environment: The environment may be fully or partially observable, deterministic or stochastic, and static or changing.
  • System organization:
    1. Centralized: A supervisor delegates tasks and combines outputs, simplifying control but creating a bottleneck.
    2. Decentralized: Peers negotiate or coordinate directly, improving resilience but increasing communication complexity.
  • Shared versus individual goals: Cooperative agents optimize a common objective; mixed-goal agents may need incentives, negotiation, or conflict resolution.
  • Key risks: Agents can duplicate work, circulate errors, deadlock while waiting, exceed tool permissions, or produce an incoherent final result.

VII. Agent Interaction — Communication and Coordination

A. Multi-agent communication and coordination mechanisms

Communication transfers information between agents, while coordination aligns their actions, timing, and resource use.

• Message contents: A structured message may contain sender, recipient, task identifier, intent, payload, deadline, and confidence; machine-readable schemas reduce ambiguity.
• Direct messaging: One agent sends a request or result to another, such as a planner assigning retrieve_policy_17 to a retrieval agent.
• Publish–subscribe: Agents publish events to topics, and interested agents subscribe; a task.completed event can trigger verification without tight coupling.
• Shared workspace: In a blackboard architecture, agents read and write plans, intermediate results, and status records in a common state store.
• Coordination patterns:
  1. Orchestration: A central controller chooses workflow order, handles retries, and synthesizes results.
  2. Choreography: Agents react to messages or events according to local rules, with no single controller directing every step.
• Protocol mechanisms: Request–response, acknowledgements, timeouts, retries, sequence numbers, and idempotency keys prevent lost or duplicated work.
• Consensus and critique: Several agents can propose solutions, compare evidence, and vote or defer to a verifier; agreement is useful but does not prove factual correctness.
• Dependency management: A directed acyclic task graph ensures that task (B) begins only after prerequisite task (A) produces its required output.
• Conflict handling: Priorities, locks, leases, or version checks prevent agents from simultaneously making incompatible updates.
• Communication cost: Excessive messages increase token use and latency; concise schemas and relevance-based routing limit unnecessary exchanges.

VIII. Task Distribution — Assigning Work to Agents

A. Multi-agent basic task allocation strategies

Task allocation maps units of work to agents according to capability, availability, cost, and dependency constraints.

• Round-robin assignment: Tasks rotate through agents in a fixed order; it is simple and fair when tasks and agents are similar but ignores specialization.
• Capability-based assignment: A router matches task requirements to declared skills, assigning SQL work to a database agent and factual validation to a verifier.
• Load-based assignment: The least-loaded eligible agent receives the next task, using measures such as queue length, estimated completion time, or active tool calls.
• Role-based decomposition: A fixed workflow assigns stages to roles—for example, planner → retriever → writer → verifier—making responsibilities clear.
• Centralized planning: A coordinator decomposes the goal, assigns subtasks, monitors status, and reallocates work after failures; global visibility improves consistency but creates a single coordination point.
• Auction-based allocation: Agents bid according to estimated cost, time, or expected quality, and the allocator selects a bid using a stated objective.
• Contract-style allocation: A manager announces a task, eligible agents submit proposals, and the manager awards and monitors the contract.
• Parallel allocation: Independent tasks run simultaneously; if retrieval takes 3 seconds and calculation takes 4 seconds, parallel execution approaches 4 seconds rather than 7, excluding overhead.
• Allocation objective:

minimize M = max(C₁, C₂, ..., Cₙ)

• (M) is the makespan, (C_i) is the completion time of agent (i), and (n) is the number of agents.
  • Practical constraints: Allocation must respect permissions, context limits, rate limits, deadlines, data locality, and prerequisite relationships.
  • Completion control: The system needs task identifiers, ownership states, acceptance criteria, retry limits, and a final integration step so that assigned work becomes one coherent result.