Unit 5: Generative and Responsible AI - Subjective Questions

Generative Artificial Intelligence refers to systems that learn patterns from existing data and use those patterns to create new content such as text, images, audio, video, software code, or synthetic data.

• Traditional rule-based systems follow explicitly programmed rules.
• Predictive Artificial Intelligence estimates an outcome or classifies existing data.
• Generative Artificial Intelligence produces novel outputs that resemble the data on which it was trained.
• Generative models may use architectures such as transformers, generative adversarial networks, variational autoencoders, or diffusion models.

For example, a spam filter predicts whether an email is spam, whereas a generative language model can compose a new email based on a prompt.

Generative Artificial Intelligence has applications across many sectors:

• Education: It can create personalized explanations, practice questions, summaries, lesson plans, and feedback.
• Healthcare: It can assist with clinical documentation, medical-image analysis, drug discovery, and patient communication, subject to professional supervision.
• Business: It can generate reports, marketing content, software code, customer-service responses, and synthetic data.
• Creative industries: It can support the creation of music, images, scripts, designs, animations, and video concepts.

These applications improve productivity and personalization, but they also require safeguards for accuracy, intellectual property, privacy, accountability, and human oversight.

A Large Language Model, or LLM, is a neural network trained on a large collection of text to understand and generate language.

• Text is divided into smaller units called tokens.
• Tokens are converted into numerical representations called embeddings.
• Most modern LLMs use the transformer architecture.
• Transformer self-attention enables the model to assign different importance to tokens in the input context.
• During training, the model commonly learns to predict the next token in a sequence.
• During inference, it generates output one token at a time according to probabilities produced by the model.

An LLM does not retrieve human-like understanding automatically. It generates statistically likely sequences based on learned patterns and the available context.

Pretraining teaches a model general language patterns using a very large and diverse dataset. The model learns vocabulary, syntax, facts, and broad relationships by solving a prediction task.

Fine-tuning further trains the pretrained model on a smaller, specialized dataset. This adapts it to a domain such as law, medicine, finance, or customer support.

Instruction tuning trains the model on examples of instructions and desirable responses. It improves the model's ability to follow user requests and produce useful, structured answers.

• Pretraining provides general capability.
• Fine-tuning provides specialization.
• Instruction tuning improves task following and interaction quality.

All three stages can inherit data-quality problems, bias, or privacy risks from their training data.

Prompt engineering is the deliberate design and refinement of inputs given to an Artificial Intelligence model to obtain accurate, relevant, and consistent outputs.

An effective prompt may contain:

• Role: The perspective or expertise the model should adopt.
• Task: A precise statement of what must be done.
• Context: Relevant background information and constraints.
• Input data: The text, question, or material to be processed.
• Output format: Requirements such as a table, list, JSON object, or explanation.
• Audience: The intended reader and expected level of detail.
• Quality criteria: Conditions such as accuracy, neutrality, citation, or completeness.

Clear prompts reduce ambiguity and help the model produce outputs that are easier to evaluate.

Important prompt design techniques include:

• Zero-shot prompting: Asking the model to perform a task without examples.
• Few-shot prompting: Providing a small number of input-output examples to demonstrate the expected pattern.
• Role prompting: Assigning an appropriate professional or analytical role.
• Stepwise decomposition: Breaking a complex task into smaller subproblems.
• Constraint prompting: Specifying limits on length, tone, sources, or content.
• Output schemas: Requiring a fixed structure such as headings, columns, or valid JSON.
• Delimiters: Separating instructions, context, and user data clearly.
• Iterative refinement: Reviewing the response and improving the prompt based on observed weaknesses.

For instance, requesting a summary with its audience, word limit, required headings, and source text is more reliable than simply asking for a summary.

• Zero-shot prompting gives only the task instruction. It is suitable for common, clearly defined tasks where the desired output is obvious.
• One-shot prompting provides one example in addition to the instruction. It is useful when the required style or interpretation needs clarification.
• Few-shot prompting provides several examples. It is appropriate for specialized classifications, formatting tasks, or cases involving subtle patterns.

The number of examples should be balanced against the available context window. Examples must be representative, correct, and free from unintended bias. Poor examples can cause the model to imitate incorrect reasoning or formatting.

Retrieval-Augmented Generation, or RAG, combines information retrieval with text generation. It gives a language model relevant external information before asking it to generate an answer.

The main stages are:

• Document preparation: Collect, clean, and divide source documents into suitable chunks.
• Indexing: Convert chunks into embeddings and store them in a searchable vector database or another retrieval system.
• Query processing: Convert the user's question into a search query or embedding.
• Retrieval: Select the most relevant document chunks.
• Augmentation: Place the retrieved content into the model's context.
• Generation: Ask the language model to answer using the supplied evidence.
• Evaluation: Check relevance, faithfulness, completeness, and citation quality.

RAG is useful when information changes frequently or must be grounded in private organizational documents.

Fine-tuning changes the model's parameters by training it on domain-specific examples. It is useful for adapting behavior, terminology, style, or task performance. However, updating knowledge requires additional training, and the model may memorize sensitive information.

RAG leaves the model parameters unchanged and supplies relevant documents at query time. It is useful for current, private, or frequently changing knowledge and can provide supporting citations.

• Fine-tuning mainly changes how the model responds.
• RAG mainly supplies what the model should use as evidence.
• Fine-tuning may improve specialized behavior.
• RAG makes knowledge updates and source inspection easier.
• A combined approach can use fine-tuning for behavior and RAG for factual grounding.

The choice depends on data freshness, privacy, cost, latency, and the desired level of traceability.

A hallucination is an output that is false, unsupported, fabricated, or misleading even though it may appear fluent and confident.

Hallucinations occur because:

• The model generates likely language rather than directly verifying truth.
• The training data may be incomplete, outdated, contradictory, or incorrect.
• The prompt may be ambiguous or provide insufficient context.
• The model may lack knowledge of a specialized or recent topic.
• Retrieval systems may provide irrelevant or incomplete evidence.
• Probabilistic decoding can produce plausible but unsupported content.

Examples include inventing a research paper, citing a nonexistent court case, attributing a quotation to the wrong person, or presenting an incorrect calculation as fact. Fluency is therefore not proof of accuracy.

Hallucination mitigation requires controls at the prompt, system, and evaluation levels:

• Use Retrieval-Augmented Generation with authoritative and current sources.
• Instruct the model to distinguish evidence from inference and to say when information is unavailable.
• Require citations, quotations, or links to supporting passages where appropriate.
• Use structured outputs and validation rules for predictable fields.
• Apply deterministic tools for arithmetic, database lookup, or code execution.
• Use a second model, rule-based checker, or human reviewer for high-risk outputs.
• Reduce unsupported creativity through suitable decoding settings.
• Test with adversarial, ambiguous, and out-of-distribution questions.
• Monitor production errors and update the knowledge base and prompts.

No single technique eliminates hallucination, so high-impact systems should use layered verification and human accountability.

Explainable Artificial Intelligence, or XAI, consists of methods that make an Artificial Intelligence system's behavior, reasoning evidence, limitations, or important influencing factors understandable to relevant people.

Explainability is important because it can:

• Support trust and informed use.
• Help developers detect errors, data leakage, and unintended shortcuts.
• Allow affected individuals to challenge or appeal decisions.
• Assist auditors and regulators in evaluating compliance.
• Improve debugging, monitoring, and system maintenance.
• Clarify when a model should not be used.

Methods include feature importance, counterfactual explanations, example-based explanations, saliency methods, model documentation, and interpretable surrogate models. Explanations should be accurate and appropriate for the audience rather than merely persuasive.

A global explanation describes how a model generally behaves across a dataset. For example, it may show that income, employment history, and repayment record are the most influential variables in a loan model overall.

A local explanation explains one particular prediction. For example, it may show why an individual loan application received a rejection based on that applicant's submitted information.

• Global explanations support model analysis, governance, and feature review.
• Local explanations support communication with an affected person and case-level investigation.
• A global explanation may not explain every individual decision.
• A local explanation may not reveal broad discriminatory patterns.

Both forms should be evaluated for faithfulness, stability, clarity, and potential to mislead.

AI bias is a systematic pattern of disadvantage, distortion, or unequal performance affecting individuals or groups. Fairness concerns whether system outcomes and processes are justifiable and do not create unjustified discrimination.

Bias may arise from:

• Historical bias: Existing social inequalities represented in the data.
• Sampling bias: Some groups being underrepresented or excluded.
• Measurement bias: Features or labels failing to measure the intended concept equally.
• Labeling bias: Subjective or inconsistent annotation decisions.
• Algorithmic bias: Model objectives or design choices producing unequal outcomes.
• Deployment bias: Using a system in a context different from the one for which it was developed.
• Feedback loops: Model decisions changing future data and reinforcing previous patterns.

Fairness must be examined throughout data collection, development, evaluation, deployment, and monitoring.

Demographic parity requires the rate of positive predictions to be similar across protected groups. In simplified form, it compares across groups, where is the prediction and is the protected attribute.

Equal opportunity requires similar true-positive rates across groups. It compares , where represents the actual positive outcome.

• Demographic parity focuses on overall selection rates.
• Equal opportunity focuses on correctly identifying qualified or eligible positive cases.
• Demographic parity may be unsuitable when groups have genuinely different base rates.
• Equal opportunity may still permit unequal false-positive rates.

Fairness criteria can conflict, particularly when groups have different base rates. Therefore, the selected criterion must reflect the social context, legal requirements, and consequences of errors.

Transparency means providing meaningful information about an AI system's purpose, development, operation, limitations, and effects so that relevant stakeholders can understand and evaluate it.

Documentation should include:

• The system's intended purpose and prohibited uses.
• Data sources, collection methods, licensing, and known limitations.
• Model type, training process, evaluation results, and performance by demographic group.
• Assumptions, uncertainty, known failure cases, and safety controls.
• Human oversight procedures and escalation paths.
• Version history, changes, access controls, and monitoring practices.
• Information about affected users' rights, feedback, and appeal mechanisms.

Useful artifacts include model cards, data sheets, risk assessments, audit logs, and system impact assessments. Transparency should protect confidential information while still enabling meaningful accountability.

Responsible Artificial Intelligence aims to ensure that AI systems are beneficial, lawful, safe, and aligned with human values. Core principles commonly include:

• Fairness: Avoid unjustified discrimination and assess performance across relevant groups.
• Transparency and explainability: Make system behavior and limitations understandable.
• Accountability: Assign clear responsibility for decisions, failures, and remediation.
• Privacy: Collect and process personal data lawfully and minimally.
• Security and safety: Resist attacks, prevent misuse, and manage operational risks.
• Human oversight: Keep people able to review, intervene, or override important decisions.
• Reliability: Test, validate, monitor, and maintain consistent performance.
• Beneficence and sustainability: Seek social benefit while reducing environmental and societal harms.

These principles should be converted into requirements, risk assessments, tests, documentation, controls, and post-deployment monitoring.

Privacy risks include exposure of personal information in prompts, memorization of sensitive training data, re-identification from generated content, unauthorized inference of private attributes, and excessive retention or sharing of user interactions.

Protection measures include:

• Collect only data necessary for the stated purpose.
• Obtain appropriate consent and provide clear privacy notices.
• Remove or mask identifying information through anonymization or pseudonymization.
• Apply access controls, encryption, secure storage, and retention limits.
• Prevent sensitive data from being used for training without authorization.
• Use privacy-preserving techniques such as differential privacy where suitable.
• Test models for memorization and data-extraction attacks.
• Establish procedures for deletion, correction, incident response, and user complaints.
• Train staff and monitor third-party AI providers.

Privacy protection should be designed into the entire data and model lifecycle.

Important security threats include:

• Prompt injection: Untrusted content attempts to override system instructions.
• Data exfiltration: The model is manipulated into revealing confidential prompts or retrieved documents.
• Sensitive information disclosure: Personal or proprietary data appears in outputs.
• Insecure tool use: The model performs harmful actions through connected tools or APIs.
• Model abuse: Attackers use the system for fraud, malware assistance, or manipulation.
• Denial of service: Excessive or specially crafted requests consume resources.
• Supply-chain risks: Compromised models, datasets, plugins, or dependencies introduce vulnerabilities.

Countermeasures include input and output filtering, strict separation of trusted instructions from untrusted data, least-privilege tool permissions, human approval for high-impact actions, rate limiting, logging, red-team testing, secret management, and continuous monitoring.

Artificial Intelligence governance is the system of policies, processes, roles, controls, and oversight mechanisms used to direct AI development and use responsibly.

An effective framework includes:

• Governance structure: Clearly assigned responsibilities for developers, owners, users, auditors, and senior decision-makers.
• Risk classification: Categorization of systems according to potential harm and impact.
• Lifecycle controls: Approval, data review, testing, deployment, monitoring, updating, and retirement procedures.
• Documentation: Records of purpose, data, model versions, tests, incidents, and decisions.
• Compliance management: Alignment with applicable laws, regulations, contracts, and organizational policies.
• Human oversight: Review and intervention procedures for high-risk use cases.
• Incident management: Reporting, investigation, corrective action, and communication processes.
• Auditing: Independent assessment of performance, security, fairness, and accountability.

Governance must remain active after deployment because model behavior and risks can change over time.