Unit 2: Exploring Large Language Models (LLMs)

I. Orientation: The Foundations Later Sections Depend On

A Large Language Model is a deep neural network trained on vast text corpora to predict the next token, and the modern generation (2017 onward) rests on the Transformer architecture introduced in "Attention Is All You Need." Every topic below — the definition, the use cases, prompt tuning, and Google's tooling — assumes the following properties.

• Transformer backbone: LLMs use stacked self-attention layers rather than recurrence; attention computes a weighted relationship between every token and every other token in the sequence.
• Tokens, not words: input text is split into sub-word units (tokens). "unbelievable" may become un, believ, able. Model size and cost are measured in tokens.
• Parameters: the learned weights. "Large" typically means billions (e.g., PaLM 2, GPT-class models). Parameter count correlates with capacity to store patterns.
• Pre-training objective: self-supervised next-token prediction over unlabeled text — no manual labels required, which is why scale is achievable.
• Foundation-model property: one pre-trained model is adapted to many downstream tasks, rather than training a fresh model per task.
• Emergence: capabilities such as arithmetic or translation appear only past a certain scale, without being explicitly programmed.

II. Defining Large Language Models

What the term precisely denotes

A. Formal Definition

An LLM is a language model with a very large parameter count, pre-trained on broad data at scale, and adaptable (via prompting or tuning) to a wide range of natural-language tasks.

• Language model, core meaning: a probability distribution over token sequences. It estimates P(token | previous tokens) and generates text by sampling repeatedly.
• "Large," quantified: the distinguishing feature versus classical n-gram or small RNN models is the combination of billions of parameters and terabyte-scale training corpora.
• General-purpose: the same weights answer questions, summarise, translate and write code — unlike task-specific classifiers.

B. How Generation Works

The model turns a prompt into output one token at a time.

• Autoregressive loop: predict next token → append it → feed the extended sequence back in → repeat until a stop token or length limit.
• Decoding controls:
  • Temperature: scales the probability distribution; low (≈0.2) is deterministic and focused, high (≈0.9) is diverse and creative.
  • Top-k / top-p: restrict sampling to the k most likely tokens, or to the smallest set whose cumulative probability exceeds p.
• Context window: the maximum tokens the model can attend to at once (prompt + output). Exceeding it truncates earlier content.

prompt: "The capital of France is"
step 1 -> "Paris"  (highest-probability next token)
output: "The capital of France is Paris"

C. Training Lifecycle

Capability is built in stages.

• Pre-training: self-supervised next-token prediction over general web/text data; produces a broad but unaligned base model.
• Fine-tuning / instruction tuning: further training on curated task or instruction data to make the model follow requests.
• RLHF (Reinforcement Learning from Human Feedback): human preference rankings train a reward model that steers outputs toward helpful, safe responses.

III. LLM Use Cases

Where the general capability delivers value

A. Orientation

A single foundation model supports many applications because most language tasks reduce to conditional text generation.

B. Content Generation and Summarisation

The most direct use — producing or condensing text.

• Generation: drafting emails, marketing copy, articles; anchored by a role prompt such as "You are a technical writer…".
• Summarisation: compressing long documents; extractive picks existing sentences, abstractive rewrites in new words (LLMs excel at the latter).

C. Conversational and Question-Answering Systems

Interactive assistants that hold context across turns.

• Chatbots: customer support agents that keep dialogue state within the context window.
• Closed vs open QA:
  1. Closed-book: answers from parameters alone — fast but risks hallucination.
  2. Open-book / RAG: retrieves documents and grounds the answer in them, reducing fabrication.

D. Code and Translation

Language modelling applies to formal languages and cross-lingual mapping.

• Code assistance: generation, completion, explanation and debugging (e.g., Codey-style models); code is just another token stream.
• Machine translation: high-quality translation without a dedicated bilingual system, since multilingual text appears in pre-training.

E. Classification and Extraction

Turning free text into structured signals.

• Classification: sentiment, intent or topic labelling, often zero-shot via a prompt like "Classify the sentiment as positive/negative."
• Information extraction: pulling entities, dates or fields from documents into JSON.

F. Applications and Limitations

• Applications summary: search, education, analytics, accessibility, and workflow automation across industries.
• Limitations: hallucination (confident false statements), knowledge cutoff (no events after training), bias inherited from data, and cost/latency at scale.

IV. Prompt Tuning

Adapting model behaviour without full retraining

A. Definition and Principle

Prompt tuning is a parameter-efficient adaptation method that learns a small set of continuous "soft prompt" vectors prepended to the input, while the model's own billions of weights stay frozen.

• Purpose: specialise a frozen foundation model to a task cheaply, storing only a tiny tuned prompt per task.
• Contrast with prompt design: prompt engineering writes natural-language instructions by hand; prompt tuning learns numeric vectors by gradient descent.

B. Soft Prompts versus Hard Prompts

The two ways to condition a model form opposed halves.

1. Hard prompts (discrete): human-written tokens like "Summarise the following:". Interpretable, require no training, but are brittle and manually optimised.
2. Soft prompts (continuous): trainable embedding vectors with no word equivalent. Not human-readable, but optimised directly against task data for higher accuracy.

Input to model = [soft_vec_1 ... soft_vec_n] + [embedded user tokens]
                  \___ learned, frozen LLM ___/
Only the soft_vec_* are updated during tuning.

• Symbols: soft_vec_i = a learned embedding of the same dimension as a token embedding; n = prompt length (a small hyperparameter, e.g., 20).

C. Prompt Tuning versus Fine-Tuning

The efficiency argument that makes prompt tuning attractive.

1. Full fine-tuning: updates all parameters; needs a full model copy per task — high storage and compute.
2. Prompt tuning: updates only the prompt vectors (a fraction of a percent of parameters); one shared frozen model serves every task, and quality approaches fine-tuning as model size grows.

D. In-Context Learning Relatives

Techniques that condition behaviour purely through the prompt at inference time.

• Zero-shot: instruction only, no examples.
• Few-shot: a handful of input-output examples placed in the prompt guide the format and task.
• Chain-of-thought: prompting the model to "think step by step" elicits intermediate reasoning and improves multi-step accuracy.

E. Significance and Limitations

• Significance: enables many task-specific deployments from one hosted model, cutting storage and serving cost.
• Limitations: soft prompts are opaque and non-transferable across models; performance gains depend on scale and still need labelled task data.

V. Google's Gen AI Development Tools

The ecosystem for building with foundation models

A. Orientation

Google offers a layered stack — from a managed ML platform down to no-code app builders — centred on the Vertex AI platform and the Gemini family of models.

B. Vertex AI and Model Garden

The enterprise platform for training, tuning and serving models.

• Vertex AI: the managed environment for the full ML lifecycle — data, tuning, deployment and monitoring — with API access to foundation models.
• Model Garden: a catalogue for discovering and deploying Google, third-party and open models from one place.
• Tuning support: offers prompt/parameter-efficient tuning and full fine-tuning against hosted foundation models.

C. Generative AI Studio and Vertex AI Studio

The interactive workspace for prompt-based development.

• Purpose: rapidly prototype and test prompts for text, chat, code and image before committing to production.
• Capabilities: side-by-side prompt iteration, decoding-parameter controls (temperature, token limit), and export of prompts to API calls.

D. Gemini and Task-Specific Models

The models the tools expose.

• Gemini: Google's flagship multimodal family, handling text, images, audio and code within one model.
• PaLM 2: an earlier large model powering text (Bison) and other variants.
• Specialised models: Codey for code, Imagen for image generation, Chirp for speech.

E. Low-Code and App-Building Tools

Options for teams that avoid direct API work.

1. Vertex AI Agent Builder / Search & Conversation: builds grounded search and chatbot experiences over an organisation's own data, reducing hallucination via retrieval.
2. AI Studio (Google AI Studio): a lightweight browser environment for prototyping with Gemini and generating API keys for developer apps.

F. Significance

• Grounding and governance: enterprise tooling adds data grounding, safety filters and access controls absent from raw model calls.
• Unified stack: one platform spans experimentation (Studio), customisation (tuning), and deployment (Vertex endpoints), shortening the path from prototype to production.