Unit 5: Generative and Responsible AI

I. Orientation — From Generating Content to Governing Its Use

Generative artificial intelligence (AI) refers to computational systems that learn patterns from data and produce new content, such as text, images, audio, video, or code. Responsible AI complements technical capability with safeguards so that AI systems remain lawful, fair, transparent, secure, privacy-preserving, and accountable throughout their life cycle.

• Core principle: Generative AI predicts or constructs outputs from learned statistical patterns; it does not possess human understanding, consciousness, or moral agency.
• Probabilistic operation: A model generally produces an output (y) in response to input (x) according to a learned probability distribution (P(y\mid x)).
• Life-cycle perspective:
  • Design: Define the purpose, users, constraints, and unacceptable harms.
  • Development: Select data, train models, evaluate performance, and document limitations.
  • Deployment: Control access, monitor outputs, protect data, and enable human oversight.
  • Retirement: Withdraw outdated systems and retain required audit records.
• Risk-based approach: Governance effort should increase with the severity and likelihood of harm; a medical decision-support system requires stronger controls than a recreational image generator.
• Human accountability: Developers, deployers, and operators remain responsible for decisions made using AI-generated outputs.

II. Generative Systems — Models, Inputs, and Grounding

A. Generative Artificial Intelligence Concepts and Applications

Generative AI learns the distribution of training data and produces new samples that resemble, recombine, or transform observed patterns.

• Model objective: A generative model estimates a distribution such as (P(x)) or conditional distribution (P(x\mid c)), where (x) is generated content and (c) is a condition such as a prompt.
• Major model families:
  • Autoregressive models: Generate one token or element at a time.
  • Generative adversarial networks: Train a generator against a discriminator.
  • Diffusion models: Learn to reverse a gradual noise-adding process, commonly for images.
  • Variational autoencoders: Encode data into a latent distribution and decode sampled representations.
• Applications: Systems generate reports, software code, product designs, synthetic training data, music, images, dialogue, translations, and personalized educational material.
• Augmentation role: In document drafting, an AI may produce a first version while a qualified person verifies facts, tone, attribution, and compliance.
• Limitations: Outputs may be inaccurate, derivative, unsafe, biased, or inconsistent because statistical plausibility is not proof of truth or originality.

B. Large Language Models

Large language models (LLMs) are neural networks, usually transformer-based, trained on extensive text collections to model relationships among tokens.

• Token prediction: A causal LLM predicts each token from preceding tokens:

P(t1, t2, ..., tn) = product from i=1 to n of P(ti | t1, ..., t(i-1))

• (t_i): the token at position (i).
• (n): the number of tokens in the sequence.
  • Transformer mechanism: Self-attention lets each token representation incorporate information from relevant tokens in its context.
  • Training stages: Pretraining develops broad language capability; instruction tuning improves task following; preference optimization can align responses with human or policy preferences.
  • Inference controls: Temperature affects randomness, while a context window limits how much prompt and generated text the model can process at once.
  • Capabilities and boundaries: LLMs can summarize, classify, translate, reason over supplied text, and generate code, but they lack guaranteed factual memory and may fail on unfamiliar or multi-step tasks.

C. Prompt Engineering Fundamentals

Prompt engineering is the systematic design and evaluation of model instructions to obtain outputs that satisfy a defined task.

• Task specification: State the required action precisely, such as “classify each review as positive, neutral, or negative.”
• Context provision: Include relevant facts, definitions, or source passages rather than expecting the model to infer missing organizational knowledge.
• Constraints: Specify audience, length, tone, permitted sources, output fields, and refusal conditions.
• Evaluation: Test prompts against representative normal, ambiguous, adversarial, and edge-case inputs; one successful response is insufficient evidence.
• Iteration cycle:

define task -> write prompt -> test cases -> inspect failures -> revise -> retest

• Security boundary: Prompt instructions cannot by themselves guarantee safety because malicious text may attempt prompt injection or request disclosure of hidden instructions.

D. Prompt Design Techniques

Prompt design techniques structure instructions so that the model receives a clear role, task, evidence base, and output contract.

• Zero-shot prompting: Gives instructions without examples; it is concise but may leave labels or formatting ambiguous.
• Few-shot prompting: Supplies representative input-output examples to demonstrate categories, style, or structure.
• Decomposition: Breaks a complex task into stages, such as extract facts, compare evidence, and then draft a conclusion.
• Structured output: A JSON schema can require predictable fields such as {"label": "...", "confidence": 0.0}, though software must still validate the result.
• Delimiters: Triple backticks or XML-style tags separate instructions from untrusted documents and reduce accidental mixing.
• Grounding instruction: Require answers to use supplied evidence, cite passage identifiers, and state when evidence is insufficient.
• Trade-off: More detailed prompts improve control but consume context space and may introduce conflicting instructions.

E. Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) combines information retrieval with generation so that an LLM answers using selected external evidence.

• Pipeline:

documents -> chunks -> embeddings -> index
query -> retrieve top-k chunks -> prompt with evidence -> generated answer

• Embeddings: A function maps text to vectors; similarity may be measured using cosine similarity:

cos(q, d) = (q · d) / (||q|| ||d||)

• (q): query vector; (d): document vector; (q\cdot d): dot product.
  • Advantages: RAG can use current, private, or domain-specific documents without retraining the base model and can attach citations.
  • Failure points: Poor chunking, stale indexes, ambiguous queries, weak ranking, or irrelevant top-(k) passages can produce unsupported answers.
  • Controls: Apply access permissions before retrieval, rerank results, preserve source metadata, and require claims to cite retrieved passages.

III. Reliability and Interpretability — Controlling Model Error

A. Hallucination in Large Language Models

Hallucination is the generation of content that is unsupported by available evidence, factually incorrect, or inconsistent with the supplied context.

• Intrinsic hallucination: The output contradicts its source, such as changing a reported revenue from £2 million to £5 million.
• Extrinsic hallucination: The output invents information absent from the source, such as a nonexistent court decision or journal article.
• Causes: Next-token prediction rewards plausible continuation; incomplete prompts, knowledge gaps, noisy training data, and retrieval failure increase error.
• Detection: Verify claims against authoritative sources, test citation existence, check calculations independently, and measure factual precision on labelled datasets.
• Risk variation: A fabricated detail in creative fiction may be harmless, while the same behavior in medication guidance can cause serious injury.

B. Hallucination Mitigation Strategies

Hallucination mitigation reduces unsupported claims through grounding, model controls, verification, and appropriate human review.

1. Preventive controls: Improve prompts, use curated RAG sources, lower randomness for factual tasks, and instruct the model to abstain when evidence is missing.
2. Detective controls: Compare generated claims with retrieved passages, run citation validation, use rule-based checks, and flag low-confidence cases.
3. Corrective controls: Regenerate with better evidence, route the case to a specialist, or block publication until verification is complete.
   • Fine-tuning role: Domain tuning may improve terminology and behavior, but it does not guarantee factual accuracy or current knowledge.
   • Human oversight: High-impact outputs require review by a person with relevant authority and expertise, not merely general familiarity with AI.
   • Evaluation metrics: Useful measures include grounded-claim rate, citation precision, abstention quality, and task-specific factual error rate.

C. Explainable Artificial Intelligence

Explainable AI (XAI) provides information that helps people understand a model’s output, relevant factors, limitations, and appropriate use.

• Global explanation: Describes overall model behavior, such as the features generally associated with loan-default predictions.
• Local explanation: Explains one result, such as which variables most influenced a particular rejected application.
• Methods: Feature importance, SHAP values, LIME approximations, counterfactual explanations, saliency maps, and interpretable decision trees provide different forms of insight.
• Counterfactual example: “Approval would change if verified annual income increased to £35,000, with other inputs fixed” communicates a decision boundary.
• Caution: An explanation may approximate behavior rather than reveal the model’s true causal reasoning; plausible text generated by an LLM is not automatically a faithful explanation.
• Audience fit: Engineers need diagnostic detail, affected individuals need understandable reasons, and auditors need reproducible evidence and documentation.

IV. Responsible AI — Rights, Controls, and Social Consequences

A. Fairness, Bias and Transparency

Fairness concerns the distribution of AI benefits, errors, and burdens, while transparency concerns meaningful disclosure of system operation and use.

• Bias sources: Historical discrimination, unrepresentative samples, proxy variables, labelling choices, and unequal deployment conditions can distort outcomes.
• Fairness measures: Demographic parity compares positive outcome rates; equal opportunity compares true-positive rates; equalized odds compares both true-positive and false-positive rates.
• Metric conflict: When groups have different base rates, multiple fairness criteria may be mathematically incompatible, so selection requires a justified policy decision.
• Bias testing: Report performance by relevant subgroups and investigate disparities rather than relying only on aggregate accuracy.
• Transparency tools: Model cards, data documentation, user notices, decision explanations, and incident reports record purpose, provenance, evaluation, and limitations.
• Meaningful disclosure: Transparency should reveal consequential facts without exposing personal data, intellectual property, or security-sensitive controls.

B. Responsible Artificial Intelligence Principles

Responsible AI principles translate ethical and legal expectations into requirements for designing, deploying, and monitoring AI.

• Beneficence: Pursue a legitimate benefit, such as improving diagnostic support or accessibility.
• Non-maleficence: Identify foreseeable harms and implement proportionate preventive controls.
• Human autonomy: Preserve informed choice, meaningful consent, contestability, and human authority over consequential decisions.
• Accountability: Assign named owners, maintain audit trails, investigate incidents, and provide remedies.
• Reliability and safety: Define operating limits, test under realistic conditions, monitor drift, and establish fallback procedures.
• Inclusiveness: Involve affected communities and test accessibility across language, disability, age, and socioeconomic contexts.
• Operationalization: Principles become effective only when connected to measurable acceptance criteria, review gates, and enforcement.

C. Privacy and Security

Privacy and security protect personal information, confidential assets, model integrity, and system availability across the AI life cycle.

• Privacy risks: Training data may contain personal data; models may memorize rare records; prompts and logs may expose confidential information.
• Privacy controls: Apply data minimization, purpose limitation, retention schedules, de-identification, access control, and privacy impact assessments.
• Technical protection: Differential privacy limits the influence of an individual record by adding controlled noise, while federated learning can reduce central collection of raw data.
• Security threats: Prompt injection, data poisoning, adversarial examples, model theft, insecure plugins, and denial-of-service attacks target different system components.
• Defense in depth: Separate untrusted content from instructions, validate tool calls, restrict privileges, encrypt data, filter outputs, and monitor abnormal activity.
• Residual risk: De-identification may be reversible when datasets can be linked, and no single prompt filter provides complete protection.

D. Artificial Intelligence Governance

AI governance is the system of policies, roles, processes, and evidence used to direct and control AI within an organization or society.

• Inventory and classification: Record each model’s purpose, owner, users, data, dependencies, and risk level.
• Stage gates: Require approval for data use, validation, deployment, major model changes, and retirement.
• Defined roles: Business owners accept operational risk; technical teams validate performance; privacy, security, legal, and ethics functions provide independent review.
• Documentation: Maintain data provenance, model versions, evaluation results, approvals, monitoring records, and incident histories.
• Monitoring: Track accuracy, subgroup performance, drift, harmful outputs, overrides, complaints, and security events after deployment.
• Governance frameworks: Risk-management standards and applicable laws provide structure, but controls must be tailored to the system’s context and consequences.

E. Ethical and Societal Implications

The ethical and societal implications of AI extend beyond model accuracy to power, labor, culture, rights, and the distribution of opportunities.

• Employment: Automation may remove tasks, create roles, and change skill requirements; impacts differ across industries and worker groups.
• Information integrity: Synthetic media can support creativity and accessibility but also enable impersonation, fraud, propaganda, and large-scale misinformation.
• Intellectual property: Training sources, generated similarities, attribution, and ownership raise legal and ethical questions for creators and deployers.
• Environmental impact: Training and serving large models consume energy, water, and hardware; efficiency and workload necessity should be assessed.
• Power concentration: Control of data, computing resources, and foundation models may concentrate economic and political influence.
• Digital inequality: Unequal connectivity, language coverage, accessibility, and AI literacy can widen existing disadvantages.
• Social response: Impact assessments, stakeholder participation, worker transition measures, content provenance, appeal mechanisms, and public accountability help align deployment with human welfare.