Unit 4: Retrieval-Augmented Generation and Multi-Agent Collaboration - Subjective Questions

Embeddings are dense numerical vectors that represent the semantic meaning of data such as words, sentences, documents, images, or queries.

• An embedding model maps an input to a vector .
• Semantically similar inputs are positioned close together in the vector space.
• During indexing, document chunks are converted into embeddings and stored in a vector index.
• At query time, the user query is embedded with a compatible model.
• The system retrieves chunks whose vectors are nearest to the query vector according to a similarity measure such as cosine similarity.

Unlike keyword search, semantic search can identify conceptual similarity even when the query and document use different words. For example, a query about reducing model fabrication may retrieve a passage discussing preventing hallucinations.

For a query embedding and document embedding , cosine similarity is defined as:

The dot product is:

The Euclidean norms are:

• A value close to indicates that the vectors point in similar directions and are likely semantically related.
• A value near indicates weak directional similarity.
• A value close to indicates opposing directions, although the practical range depends on the embedding model.
• Documents are commonly ranked in descending order of similarity.

Cosine similarity focuses on vector direction rather than magnitude. If embeddings are normalized to unit length, cosine similarity simplifies to the dot product.

Sparse retrieval represents text using high-dimensional lexical features.

• Techniques include TF-IDF and BM25.
• It is effective for exact terms, identifiers, rare names, and domain-specific keywords.
• It may miss relevant passages that express the same idea with different vocabulary.

Dense retrieval represents queries and documents using learned embeddings.

• It captures semantic and contextual similarity.
• It handles paraphrases well.
• It can perform poorly on exact codes, uncommon proper nouns, or vocabulary not represented well during training.

Hybrid retrieval combines sparse and dense results.

• It may use a weighted score such as after score normalization.
• Alternatively, ranked lists can be combined using reciprocal rank fusion.
• It is useful when both exact lexical matching and semantic understanding are important.

Sparse retrieval is suitable for keyword-heavy corpora, dense retrieval for concept-oriented search, and hybrid retrieval for robust production systems containing mixed query types.

Fixed-size chunking divides content by a fixed number of characters, words, or tokens.

• It is simple, predictable, and inexpensive.
• It may split sentences, tables, or related ideas at arbitrary positions.

Recursive chunking repeatedly splits text using an ordered set of separators, such as sections, paragraphs, sentences, and spaces.

• It preserves natural text boundaries better than fixed-size chunking.
• It remains straightforward to implement.
• Its quality depends on document structure and separator rules.

Semantic chunking groups sentences or passages according to topic or embedding similarity.

• It creates conceptually coherent chunks.
• It can improve retrieval precision for complex documents.
• It requires additional computation and threshold tuning.

The appropriate strategy depends on document structure, cost, and retrieval goals. Structured technical documents often benefit from recursive chunking, while topic-rich narrative content may benefit from semantic chunking.

Chunk size controls how much information is stored in each retrieval unit.

• Small chunks usually improve retrieval precision because each chunk covers a narrow topic.
• However, small chunks may omit necessary context and increase the number of vectors.
• Large chunks preserve broader context but may include irrelevant information, reduce precision, and consume more prompt tokens.

Chunk overlap repeats boundary content between adjacent chunks.

• It prevents important sentences or relationships from being lost at chunk boundaries.
• Excessive overlap creates duplicate retrieval results, increases storage, and wastes context-window capacity.
• Insufficient overlap can separate definitions from explanations or headings from their content.

A good configuration is selected empirically. It should consider the embedding model's input limit, typical answer scope, document structure, retrieval latency, vector-storage cost, and the language model's context window.

An effective indexing strategy should preserve both semantic content and document structure:

1. Ingest and normalize: Extract text while preserving headings, table boundaries, code blocks, and document identifiers.
2. Clean carefully: Remove repeated navigation or boilerplate without deleting meaningful content.
3. Use content-aware chunking: Split reports by sections, web pages by headings, tables by logical rows or groups, and code by functions or classes.
4. Attach metadata: Store source, title, section, timestamp, content type, access permissions, language, and chunk position.
5. Create embeddings: Use a model suitable for the domain and apply consistent preprocessing to documents and queries.
6. Build complementary indexes: Maintain a vector index for semantic retrieval and a lexical index for exact terms.
7. Preserve relationships: Link chunks to parent documents and neighboring chunks for later context expansion.
8. Support updates: Track document versions and remove obsolete vectors when sources change.

This strategy enables filtered, traceable, and structure-aware retrieval across different data formats.

A vector database is a data system optimized for storing, indexing, filtering, and retrieving high-dimensional vectors.

Its main components include:

• Vector storage: Stores embeddings and associates them with chunks or records.
• Similarity index: Uses exact or approximate nearest-neighbor algorithms to locate related vectors efficiently.
• Metadata store: Maintains fields such as source, date, category, tenant, and permissions.
• Filtering engine: Restricts search to records satisfying metadata conditions.
• Persistence and scaling layer: Supports replication, partitioning, durability, and distributed search.

Typical operations are:

• Upsert: Insert or update a vector and its metadata.
• Delete: Remove obsolete or unauthorized entries.
• Query: Return the top- nearest vectors.
• Filter: Apply metadata constraints before or during search.
• Fetch: Retrieve a record directly by identifier.

In RAG, the database connects an embedded user query to relevant knowledge chunks that are supplied to the language model.

Exact nearest-neighbor search compares a query with every stored vector.

• It provides exact results under the chosen distance measure.
• Its straightforward time cost is approximately for vectors of dimension .
• It becomes expensive for very large collections.

Hierarchical Navigable Small World graphs (HNSW) organize vectors in a multilayer proximity graph.

• Search begins in sparse upper layers and moves toward the query through increasingly dense layers.
• It generally provides high recall and low query latency.
• It can require significant memory and has construction-time costs.

Inverted File Index (IVF) clusters vectors and assigns them to lists based on nearby centroids.

• At query time, only selected clusters are searched.
• It reduces the number of comparisons and can scale well.
• Recall depends on clustering quality and the number of probed clusters.

Approximate methods trade a small amount of recall for major gains in speed and scalability. Their parameters should be tuned against latency, memory usage, and retrieval recall.

A complete RAG pipeline has two main phases.

Offline ingestion and indexing:

1. Collect documents from trusted sources.
2. Parse, clean, and normalize the content.
3. Divide documents into suitable chunks.
4. Attach metadata and access-control information.
5. Generate embeddings for the chunks.
6. Store vectors, text, metadata, and source references in searchable indexes.

Online retrieval and generation:

1. Receive and validate the user query.
2. Apply query rewriting, decomposition, or expansion if needed.
3. Generate the query embedding.
4. Retrieve candidate chunks using dense, sparse, or hybrid search.
5. Apply metadata and permission filters.
6. Rerank, deduplicate, and optionally expand the best chunks with neighboring context.
7. Construct a prompt containing instructions, retrieved evidence, and the query.
8. Generate an answer that is constrained by the supplied evidence.
9. Attach citations and perform grounding or confidence checks.

The pipeline should also log retrieval results and feedback for evaluation without exposing sensitive content.

Query rewriting converts an ambiguous or conversational request into a clear, standalone search query. It is especially useful when the original query depends on chat history.

Query expansion adds synonyms, abbreviations, related terms, or generated alternative queries. It improves recall when relevant documents use different vocabulary.

Query decomposition divides a complex question into smaller subqueries. Each subquery retrieves evidence for one part of the reasoning process.

For example, a question asking whether one system is cheaper and faster than another can be decomposed into separate cost and latency queries.

These methods can improve retrieval by:

• Resolving references and ambiguity.
• Bridging vocabulary differences.
• Retrieving evidence for multiple constraints.
• Supporting multi-hop reasoning.

However, poor transformations may change the user's intent or introduce unsupported assumptions. Systems should retain the original query, limit expansions, deduplicate results, and verify that rewritten queries remain semantically aligned.

Reranking is the process of rescoring an initial set of retrieved candidates to produce a more relevant final ordering.

A bi-encoder independently encodes the query and each document into vectors.

• Document embeddings can be precomputed.
• Retrieval is fast and scalable.
• Query-document interaction is compressed into a vector similarity score.

A cross-encoder processes the query and candidate document together.

• It can model detailed token-level interactions.
• It generally produces more accurate relevance scores.
• It is computationally expensive because every query-candidate pair must be evaluated.

A common pipeline retrieves a relatively broad set such as top- candidates with a bi-encoder, reranks those candidates with a cross-encoder, and sends only the best few passages to the generator. This combines scalable recall with stronger final precision.

Grounding requires the generated answer to be supported by retrieved evidence rather than relying only on the model's internal parameters.

• The prompt should clearly separate instructions, evidence, and the user's question.
• It should tell the model to use only supported claims and acknowledge insufficient evidence.
• Relevant, authoritative, and current chunks should be prioritized.

Citations connect claims to their sources.

• Stable document and chunk identifiers should be retained throughout retrieval.
• Citations improve traceability and allow users to verify claims.
• Citation presence alone is insufficient; each citation must actually entail the associated statement.

Context management removes duplicates and irrelevant passages, respects the context-window limit, and preserves necessary neighboring information.

Additional safeguards include entailment checks, conflict detection, source-quality scoring, structured answer formats, and abstention rules. RAG reduces hallucination risk but does not eliminate it, because retrieval may fail and the model may still misinterpret valid evidence.

RAG evaluation should separate retrieval quality from generation quality.

Retrieval metrics:

• Recall@: Fraction of relevant items found among the top results.
• Precision@: Fraction of the top results that are relevant.
• Mean Reciprocal Rank: Uses the rank of the first relevant result, with score .
• nDCG: Rewards relevant items more when they appear near the top and can account for graded relevance.

Generation metrics:

• Faithfulness: Whether claims are supported by retrieved evidence.
• Answer relevance: Whether the response addresses the question.
• Correctness and completeness: Whether the required facts are accurate and sufficiently covered.
• Citation accuracy: Whether citations support the claims linked to them.

Operational and end-to-end measures:

• Task success, abstention quality, latency, token usage, cost, freshness, and user satisfaction.
• Performance should be tested on representative queries, including ambiguous, adversarial, multi-hop, and unanswerable cases.

Human evaluation and carefully reviewed benchmark datasets remain important because automated model-based judges can introduce bias.

A multi-agent system (MAS) consists of multiple autonomous or semi-autonomous agents that interact within a shared environment to achieve individual or collective goals.

Important characteristics include:

• Autonomy: Each agent can observe, decide, and act with limited direct control.
• Specialization: Agents may have different roles, tools, knowledge, or capabilities.
• Interaction: Agents exchange messages, artifacts, or environmental signals.
• Coordination: Their actions are organized to avoid conflicts and support shared objectives.
• Distributed control: Decision-making may be divided rather than concentrated in one component.
• Adaptability: Agents can revise plans in response to feedback or environmental changes.
• Emergent behavior: System-level outcomes can arise from local interactions.

In agentic AI, agents may be implemented with language models, memory, planning modules, tools, and policies. Multiple agents can improve modularity and parallelism, but they also introduce communication overhead, inconsistency, and more complex failure modes.

Centralized architecture:

• A coordinator assigns tasks, maintains global state, and combines results.
• It simplifies control, monitoring, and conflict resolution.
• The coordinator may become a bottleneck or single point of failure.

Decentralized architecture:

• Agents communicate directly and make local decisions.
• It improves resilience and can support parallel activity.
• Achieving consistent state and coordinated decisions is more difficult.

Hierarchical architecture:

• A top-level agent decomposes objectives, while intermediate managers and worker agents handle progressively narrower tasks.
• It supports scalable delegation and clear responsibilities.
• Errors or distorted instructions can propagate through the hierarchy.

A suitable design depends on task coupling, reliability requirements, communication cost, and scale. Many practical systems are hybrid: a coordinator defines objectives and permissions while specialized agents collaborate directly for selected subtasks.

Major communication mechanisms include:

• Direct messaging: One agent sends a request or result to another. It is appropriate for explicit dependencies and targeted collaboration.
• Broadcast or publish-subscribe: Messages are published to a topic and received by interested agents. It supports loose coupling and event-driven workflows.
• Shared memory or blackboard: Agents read and write plans, facts, and partial results in a common workspace. It is useful when many specialists contribute to one evolving solution.
• Environment-mediated communication: Agents coordinate indirectly by changing observable environmental state. This is useful in simulations and distributed control.
• Structured protocols: Agents communicate through typed messages, schemas, state machines, or agreed performatives such as request, proposal, acceptance, and rejection.

Effective messages should include sender, recipient, task identifier, intent, payload, timestamp, confidence, provenance, and expected response. Structured communication reduces ambiguity, while validation, authentication, access control, and timeout handling improve reliability and security.

Multi-agent coordination can use several complementary mechanisms:

• Shared planning: Agents contribute actions to a common plan with explicit goals, dependencies, and deadlines.
• Role assignment: Responsibilities and authority are defined to reduce duplicated or contradictory work.
• Synchronization: Barriers, events, locks, or dependency graphs ensure that actions occur in a valid order.
• Negotiation: Agents exchange proposals and revise choices when goals or resource demands conflict.
• Consensus or voting: Agents select among alternatives when no single agent has final authority.
• Arbitration: A designated agent resolves unresolved disputes.

Consistent shared state can be supported through version numbers, immutable event logs, transactional updates, idempotent operations, and explicit ownership. Agents should detect stale information and retry safely after failures. Complete consistency may be expensive in distributed systems, so designers must choose an appropriate balance among consistency, availability, latency, and autonomy.

Centralized allocation uses a manager or scheduler to collect task requirements and agent capabilities before assigning work.

• It can optimize globally when accurate information is available.
• It supports priorities, deadlines, and resource constraints.
• It creates communication overhead and a potential single point of failure.

Decentralized allocation lets agents select, claim, negotiate, or exchange tasks using local information.

• It is resilient and adaptable to dynamic environments.
• It avoids dependence on one coordinator.
• It may produce duplicate effort, uneven load, or locally optimal decisions.

Basic allocation criteria include capability match, expected completion time, workload, cost, proximity, reliability, and dependency constraints. A practical hybrid system may centrally allocate critical tasks while allowing agents to self-organize routine or rapidly changing work.

The Contract Net Protocol allocates tasks through a manager-contractor negotiation process:

1. Task announcement: A manager broadcasts a task description, eligibility conditions, evaluation criteria, and deadline.
2. Bidding: Eligible agents evaluate their capabilities and current workload, then submit proposals.
3. Award: The manager compares bids and awards the task to the most suitable agent.
4. Execution: The selected contractor performs the task and reports progress or exceptions.
5. Completion: The contractor returns the result, which the manager verifies and integrates.

Strengths:

• Supports dynamic capability-based allocation.
• Distributes decision-making among agents.
• Can account for cost, time, quality, and workload.

Limitations:

• Announcements and bids create communication overhead.
• Agents may estimate costs inaccurately.
• Delayed bids can slow execution.
• Locally attractive awards may not produce a globally optimal schedule.

Timeouts, bid validation, reputation scores, re-auctioning, and task cancellation rules can make the protocol more robust.

A multi-agent RAG system can assign specialized roles:

• Coordinator agent: Interprets the objective, decomposes it into subquestions, assigns tasks, and tracks dependencies.
• Query-planning agent: Rewrites and expands subqueries.
• Retriever agents: Search different indexes, domains, or data modalities in parallel.
• Reranker agent: Scores, deduplicates, and filters candidate evidence.
• Reasoning or synthesis agent: Combines evidence into a coherent draft.
• Verifier agent: Checks factual entailment, contradictions, and citation coverage.
• Response agent: Produces the final answer in the required format.

A typical flow is:

1. Decompose the research question into independently retrievable parts.
2. Allocate subqueries according to agent expertise and data access.
3. Retrieve evidence with permission and metadata filters.
4. Store results in a shared workspace with provenance and confidence scores.
5. Detect duplicated or conflicting sources.
6. Synthesize only after required evidence dependencies are satisfied.
7. Verify each important claim against cited passages.

Safeguards should include least-privilege tool access, schema-validated messages, prompt-injection filtering, source trust policies, bounded iteration, timeouts, audit logs, and abstention when evidence is insufficient. This design improves specialization and parallelism, although coordination cost must be justified by task complexity.