Unit 1: Foundations of LLMs and Agentic Systems

I. Orientation — From Language Models to Autonomous Agents

Large language models (LLMs) are neural networks trained to process and generate language, while agentic systems place models inside decision-making loops that pursue goals through observation, reasoning, action, and feedback. Modern agentic AI combines transformer-based models (introduced in 2017) with tools, memory, planning, and environmental interaction.

• Governing principle: An LLM estimates patterns in token sequences; an intelligent agent uses such estimates, alongside state and goals, to select actions.
• Core components:
  • Model: Maps an input token sequence to probability distributions or representations.
  • Agent: Chooses actions according to observations, goals, and an internal policy.
  • Environment: Supplies observations and changes in response to actions.
  • Tools: Extend the agent through search, calculators, databases, code execution, or APIs.
• Central distinction: A standalone LLM generates responses, whereas an agentic system repeatedly observes results and decides what to do next.
• Learning assumption: Statistical regularities learned from large datasets can support generalization to previously unseen prompts and tasks.
• Operational constraint: Fluent output does not guarantee factual correctness, reliable reasoning, safe action, or genuine understanding.

II. Intelligent Agents — Goal-Directed Decision-Making

A. Introduction to intelligent agents and their types

An intelligent agent is an entity that perceives an environment through sensors or inputs and acts upon it through actuators or outputs to achieve an objective.

• Agent function: The abstract mapping from observation history to action is:

a_t = π(o_1, o_2, ..., o_t)

• (a_t): action selected at time (t).
• (\pi): agent policy or decision rule.
• (o_t): observation received at time (t).

• Rational agent: Selects the action expected to maximize a performance measure given its observations and available knowledge; rationality means justified choice, not omniscience.
• Simple reflex agent: Uses condition-action rules such as IF temperature > 30°C THEN start_fan; it works best in fully observable environments.
• Model-based reflex agent: Maintains an internal state when the current observation is incomplete, such as a robot tracking obstacles no longer visible.
• Goal-based agent: Evaluates actions by whether they lead toward a specified target, such as finding a route from node (A) to node (B).
• Utility-based agent: Compares outcomes using a utility function (U(s)), allowing trade-offs among speed, cost, safety, and quality.
• Learning agent: Improves from experience through:
  • Performance element: Chooses actions.
  • Learning element: Updates behavior.
  • Critic: Evaluates outcomes.
  • Problem generator: Encourages useful exploration.
• LLM-based agent: Uses an LLM as a reasoning or control component, often supplemented by memory, planning modules, and external tools.

III. Agentic AI — The Shift from Generation to Action

A. Emergence of Agentic AI

Agentic AI emerged from combining capable foundation models with mechanisms that let systems pursue multistep goals rather than produce only one-shot responses.

• Enabling development: Transformer scaling produced models able to follow instructions, generate plans, write code, and interpret heterogeneous inputs.
• Instruction tuning: Training on prompt-response examples makes pretrained models more responsive to human commands and task formats.
• Preference alignment: Human or AI feedback can optimize outputs toward criteria such as helpfulness, safety, and relevance.
• Tool augmentation: Function calling allows a model to request a structured operation, for example:

{"tool": "calculator", "arguments": {"expression": "17 * 24"}}

• Agentic workflow: A typical system decomposes a goal, chooses tools, observes results, updates its plan, and stops when a completion condition is met.
• Degrees of agency: Agency ranges from approval-based assistants to systems that independently schedule and execute several actions.
• Benefits: Agents can automate research, software workflows, customer support, data analysis, and document processing.
• Limitations: Error propagation, prompt injection, excessive tool permissions, hallucinated plans, and runaway costs require sandboxing, monitoring, and human approval.

IV. Transformers — Architecture for Sequence Modelling

A. Transformer architecture basics

The transformer is a neural architecture that processes token relationships primarily through attention rather than recurrence, enabling efficient parallel training.

• Input representation: Each token receives an embedding vector combined with positional information so the model can distinguish order.
• Transformer block: A standard block contains:
  • Multi-head attention: Learns several relationship patterns in parallel.
  • Feed-forward network: Applies nonlinear transformations independently at each position.
  • Residual connections: Add a sublayer’s input to its output, improving gradient flow.
  • Layer normalization: Stabilizes hidden activations during training.
• Encoder-only design: Models such as BERT build bidirectional representations and are suited to classification, extraction, and embedding tasks.
• Decoder-only design: GPT-style models use causal masking so each position attends only to earlier tokens; this supports autoregressive generation.
• Encoder-decoder design: Models such as T5 encode an input sequence and decode a separate output sequence, fitting translation and summarization.
• Generation rule:

P(x_1, ..., x_n) = ∏ P(x_t | x_1, ..., x_(t-1))

• (x_t): token at position (t).
• (n): sequence length.
• (P(xt \mid x{<t})): probability of the next token given preceding tokens.
  • Limitation: Standard self-attention has approximately (O(n^2)) time and memory cost with sequence length (n).

V. Attention — Selecting Relevant Context

A. Attention mechanism

Attention computes how strongly each token should incorporate information from other permitted tokens in the sequence.

• Query, key, and value: Hidden vectors are projected into queries (Q), keys (K), and values (V); queries seek information, keys identify it, and values carry its content.
• Scaled dot-product attention:

Attention(Q, K, V) = softmax(QKᵀ / √d_k)V

• (QK^\top): similarity scores between queries and keys.
• (d_k): key-vector dimension; scaling by (\sqrt{d_k}) controls score magnitude.
• softmax: converts each score row into weights summing to 1.
  • Self-attention: (Q), (K), and (V) come from the same sequence, allowing “bank” to relate differently to “river” and “money.”
  • Cross-attention: Decoder queries attend to encoder outputs, linking generated tokens to source information.
  • Multi-head attention: Several heads can specialize in positional, syntactic, semantic, or reference relationships.
  • Causal mask: Decoder-only models assign inaccessible future positions effectively zero attention weight, preserving next-token prediction.
  • Interpretive caution: Attention weights show computational routing but do not, by themselves, provide a complete explanation of model reasoning.

VI. Tokenization — Converting Text into Model Inputs

A. Tokenization approaches

Tokenization converts raw text into discrete token identifiers that a model can embed and process.

• Word tokenization: Splits text into words; it is intuitive but produces large vocabularies and handles unseen words poorly.
• Character tokenization: Represents individual characters; its vocabulary is small, but sequences become long and semantic units are fragmented.
• Subword tokenization: Represents frequent words directly and rare words as pieces, balancing vocabulary size with sequence length.
• Byte Pair Encoding: Repeatedly merges frequent adjacent symbols; “unhappiness” might become un, happi, and ness.
• WordPiece: Selects subword units using a likelihood-oriented criterion and is associated with BERT-style tokenizers.
• Unigram model: Begins with many candidate pieces and removes those whose loss least harms the tokenization model; SentencePiece supports this approach.
• Byte-level tokenization: Operates over byte representations, reducing unknown-token problems across languages and unusual symbols.
• Vocabulary mapping: A tokenizer produces IDs such as [415, 927, 13], not semantic vectors; the embedding table maps those IDs into vectors.
• Practical effect: Token boundaries influence context usage, computational cost, multilingual performance, and how reliably a model handles numbers or code.

VII. LLM Training — Learning from Large Corpora

A. Pre-training objectives in LLMs

Pre-training teaches general linguistic and statistical patterns through self-supervised objectives constructed directly from large text collections.

• Causal language modelling: Predicts the next token from earlier tokens:

L = -Σ log P(x_t | x_<t)

• (L): negative log-likelihood loss.
• (x_t): target token.
• (x_{<t}): tokens preceding position (t).

• Masked language modelling: Replaces selected input tokens with masks and predicts the originals; BERT can therefore use context on both sides.
• Denoising objective: Corrupts text by deleting, masking, or rearranging spans and trains the model to reconstruct the clean sequence.
• Sequence-to-sequence objective: Encodes a corrupted or source sequence and generates a target sequence, as in T5-style pre-training.
• Self-supervision: Labels come from the text itself, avoiding manual annotation for every training example.
• Training result: The model acquires reusable representations, factual associations, stylistic patterns, and task-relevant capabilities.
• Post-training distinction: Supervised fine-tuning and preference optimization adapt pretrained behaviour but are not the original pre-training objective.
• Limitations: Corpus bias, duplicated data, privacy concerns, stale facts, and prediction-focused objectives can produce confident but unsupported claims.

VIII. Agent Control — Repeated Observation and Action

A. Agent perception-action cycle

The perception-action cycle is the iterative process through which an agent observes its situation, decides, acts, and evaluates the resulting state.

• Perceive: Collect observations such as user messages, tool results, sensor readings, files, or API responses.
• Interpret: Convert observations into an internal state (s_t), possibly using retrieval, memory, or structured parsing.
• Plan: Select an intermediate objective or action sequence consistent with the overall goal.
• Act: Execute action (a_t), such as sending a command, querying a database, or producing a response.
• Evaluate: Compare the outcome with success criteria and identify errors or missing information.
• Update: Store relevant results and revise the state or plan before another cycle.

observe → update state → choose action → execute → evaluate → repeat/stop

• Stopping condition: The cycle ends when the goal is satisfied, a step budget is exhausted, confidence is insufficient, or human authorization is required.
• Risk control: High-impact actions should use permission boundaries, validation checks, audit logs, and reversible operations.

IX. Environments — Context, Feedback, and Consequences

A. Agent-environment interaction

Agent-environment interaction describes how actions alter external conditions and how resulting observations shape later decisions.

• Environment model: Interaction can be represented as a state transition:

s_(t+1) ~ T(s_(t+1) | s_t, a_t)

• (s_t): environment state at time (t).
• (a_t): selected action.
• (T): transition function or probability distribution.

• Fully versus partially observable: A chess program sees the full board, while a support agent may lack information about a customer’s unstated constraints.
• Deterministic versus stochastic: A calculator call is usually deterministic; web traffic or physical movement may have uncertain outcomes.
• Static versus dynamic: A fixed document remains unchanged during reasoning, whereas financial or robotic environments may change continuously.
• Discrete versus continuous: Board moves are discrete; vehicle steering and time are often continuous.
• Feedback: Rewards, errors, user responses, and tool outputs help the agent evaluate whether an action advanced its goal.
• Grounding: Retrieval and tools connect model output to current external evidence, but unreliable sources or malformed tool results can still mislead the agent.
• Safety boundary: Least-privilege access limits consequences by granting only the tools and data required for the current task.

X. Major Model Families — Representative LLM Ecosystems

A. Overview of popular LLMs

Popular LLMs differ in architecture, access model, context capacity, modality, deployment options, and specialization.

• GPT family: OpenAI’s decoder-oriented models emphasize general generation, instruction following, coding, multimodal interaction, and tool use.
• Claude family: Anthropic’s models emphasize long-context processing, dialogue, analysis, coding, and safety-oriented deployment.
• Gemini family: Google DeepMind develops multimodal models integrated with Google’s cloud, productivity, and developer ecosystems.
• Llama family: Meta releases openly available model weights under specified licences, supporting local adaptation, research, and deployment.
• Mistral and Mixtral: Mistral AI provides efficient models, including mixture-of-experts systems that activate only selected parameter groups per token.
• Qwen family: Alibaba’s models cover multilingual, coding, mathematical, vision-language, and agent-oriented use cases.
• DeepSeek family: DeepSeek develops models emphasizing coding, reasoning, and computational efficiency, including mixture-of-experts designs.
• BERT family: Encoder-only BERT is not primarily a conversational generator; it remains influential for classification, retrieval, and representation learning.
• Selection criteria:
  • Quality: Accuracy on the target language, domain, and task.
  • Operations: Latency, context limits, throughput, hardware, and cost.
  • Governance: Licensing, privacy, data residency, safety controls, and auditability.
• Evaluation principle: No model is universally best; controlled tests on representative data are more reliable than selecting solely by parameter count or benchmark rank.