Unit 1: Foundations & Applications of AI; AI Problem Modeling & Search Concepts

I. Orientation: Intelligence and Artificial Intelligence

Artificial intelligence rests on a claim that reasoning behaviour observed in humans can be described precisely enough to be reproduced by a machine. This section fixes the two definitions the rest of the unit depends on.

A. What is Intelligence

Intelligence is the capacity to acquire knowledge and apply it to achieve goals in varied, changing environments.

• Adaptation: adjusting behaviour when conditions change rather than repeating fixed responses.
• Learning: improving performance from experience or data (a child recognising cats after few examples).
• Reasoning: deriving new conclusions from known facts (if all A are B, and x is A, then x is B).
• Perception and language: interpreting sensory input and communicating symbolically.

B. What is AI

AI is the branch of computer science that builds systems performing tasks that would require intelligence if done by humans.

• Operational definition (Turing, 1950): a machine is intelligent if its responses are indistinguishable from a human's in text conversation.
• Four views (Russell & Norvig): systems that think like humans, act like humans, think rationally, and act rationally — modern AI favours the rational agent view: maximise expected performance.
• Agent model: perceive environment via sensors → decide → act via actuators.

II. Characteristics and Types of AI

A. Characteristics of artificial intelligence

The traits below distinguish AI systems from ordinary deterministic programs.

• Learning from data: parameters are fit to examples, not hand-coded.
• Reasoning and inference: deriving decisions from rules or probabilities.
• Perception: processing images, audio, and text as input.
• Handling uncertainty: producing best guesses under incomplete information using probability.
• Autonomy and goal-orientation: acting toward objectives with limited supervision.
• Generalisation: performing on unseen inputs, not just memorised ones.

B. Types of AI (narrow, general)

AI is classified by breadth of capability.

1. Narrow AI (ANI): excels at one task and cannot transfer skill. Examples: spam filters, chess engines, recommendation systems. All AI deployed today is narrow.
2. General AI (AGI): hypothetical system matching human flexibility across any intellectual task, transferring knowledge between unrelated domains. Not yet achieved.

• Superintelligence (ASI): a speculative stage exceeding human ability in all fields; theoretical only.

III. Foundations and Evolution of AI

A. Foundations of AI

AI draws its concepts from several established disciplines.

• Philosophy: logic and reasoning; can thought be formalised (Aristotle's syllogisms).
• Mathematics: logic, probability, and computability (Boolean algebra, Bayes' theorem).
• Economics: decision theory and utility maximisation under constraints.
• Neuroscience: the brain as an information-processing organ inspiring neural networks.
• Psychology: models of perception and cognition.
• Linguistics: structure of language, underpinning NLP.
• Computer engineering: hardware and algorithms that make computation feasible.

B. Evolution

AI has progressed through distinct phases.

• 1943–1955 (Gestation): McCulloch–Pitts artificial neuron; Turing's foundational questions.
• 1956 (Birth): Dartmouth Workshop coins the term "artificial intelligence".
• 1956–1974 (Early enthusiasm): symbolic reasoning, search-based problem solvers.
• 1974–1980 & late 1980s (AI Winters): funding collapses as promises outpace results.
• 1980s (Expert systems): rule-based commercial systems like MYCIN.
• 1997: Deep Blue defeats chess champion Kasparov.
• 2012 onward (Deep learning era): AlexNet's ImageNet win triggers the neural-network revolution; large datasets and GPUs enable transformers and generative models.

IV. Key AI Problems and Techniques

AI research organises around recurring problem classes and the tool families used to solve them.

A. Key AI problems

• Search and planning: finding a path or action sequence to a goal (route finding, scheduling).
• Knowledge representation: encoding facts so machines can reason (ontologies, logic).
• Reasoning under uncertainty: drawing conclusions from noisy data (medical diagnosis).
• Learning: inferring patterns from examples (classification, regression).
• Perception: interpreting vision and speech signals.
• Natural language understanding: parsing meaning from text.

B. AI techniques

• Search algorithms: BFS, DFS, A* for state-space exploration.
• Logic and rule-based systems: if–then inference engines.
• Probabilistic models: Bayesian networks, hidden Markov models.
• Machine learning: supervised, unsupervised, and reinforcement learning.
• Neural networks / deep learning: layered models for perception and language.

V. Applications across domains

AI is applied wherever pattern recognition or decision automation adds value.

• Business: fraud detection scoring transactions in real time; demand forecasting; recommendation engines (product suggestions from purchase history).
• Healthcare: tumour detection in radiology images; drug-discovery molecule screening; clinical decision support estimating risk.
• Automation: robotic process automation for repetitive back-office tasks; industrial robots on assembly lines; self-driving vehicle control.
• Vision: face verification, optical character recognition, defect inspection in manufacturing.
• Language: machine translation, chatbots, sentiment analysis, speech-to-text transcription.

VI. Modern AI Toolkits (TensorFlow, PyTorch)

These frameworks provide the tensor operations and automatic differentiation needed to build neural networks.

1. TensorFlow (Google, 2015): production-oriented; originally static computation graphs, now supports eager execution; strong deployment tooling (TensorFlow Serving, TF Lite for mobile).
2. PyTorch (Meta, 2016): dynamic ("define-by-run") graphs favoured in research for flexible debugging; dominant in academic publications.

• Shared core: both expose GPU-accelerated tensors and reverse-mode autodiff.

# PyTorch: one gradient step
import torch
x = torch.tensor([1.0], requires_grad=True)
loss = (x - 3) ** 2      # simple quadratic loss
loss.backward()          # autodiff computes dloss/dx = 2(x-3) = -4
# x.grad now holds the gradient used to update x

• Symbols: requires_grad tracks the tensor for differentiation; backward() populates .grad with derivatives.

VII. Responsible AI

Responsible AI is the practice of designing systems that are fair, transparent, and accountable.

• Fairness: avoiding bias against groups; a hiring model must not penalise gender inferred from data.
• Transparency / explainability: making decisions interpretable (why a loan was denied).
• Accountability: assigning human responsibility for automated outcomes.
• Privacy: protecting personal data (anonymisation, consent).
• Safety and robustness: reliable behaviour under adversarial or edge-case input.
• Sources of harm: biased training data, feedback loops, and opaque "black-box" models.

VIII. Defining AI Problems as State Space and Search Problems

A large class of AI tasks is reframed as searching a space of configurations for a goal.

A. Defining AI problems as State Space and Search Problems

A problem is specified as a formal five-part tuple.

• Initial state: the starting configuration (start city).
• Actions / operators: legal moves transforming one state to another.
• Transition model: result of applying an action to a state.
• Goal test: predicate identifying a solution state.
• Path cost: numeric measure to minimise (distance, time).

State space = { all states reachable from the initial state via operators }
Solution     = a sequence of actions from initial state to a goal state

• Worked example (8-puzzle): state = tile arrangement; operators = slide blank up/down/left/right; goal = ordered tiles; cost = number of moves.

B. Characteristics of AI Problem Spaces

Problem spaces vary along dimensions that determine which methods apply.

• Size: often exponential; chess has ~10⁴⁷ legal positions, ruling out exhaustive search.
• Observability: fully observable (chess) versus partially observable (poker).
• Determinism: deterministic outcomes versus stochastic ones (dice).
• Discreteness: discrete states versus continuous ones (robot joint angles).
• Static vs dynamic: does the environment change while deliberating.
• Decomposability: whether the problem splits into independent subproblems.

C. Introduction to AI Workflows & Data-Centric Modeling

Modern AI development follows a repeatable pipeline centred on data quality.

• Workflow stages:
  • Problem definition: frame the objective and success metric.
  • Data collection and preparation: gather, clean, and label data.
  • Feature engineering: derive informative inputs.
  • Model training: fit parameters to data.
  • Evaluation: test on held-out data (accuracy, precision, recall).
  • Deployment and monitoring: serve predictions and watch for drift.
• Data-centric modeling: improve results by systematically raising data quality (fixing labels, balancing classes) rather than only tweaking model architecture — contrasts with the older model-centric emphasis.

D. Challenges in AI problem solving

Building working AI systems is constrained by recurring practical difficulties.

• Combinatorial explosion: search spaces grow faster than compute, forcing heuristics.
• Data scarcity and quality: insufficient, noisy, or biased data caps performance.
• Overfitting: memorising training data instead of generalising.
• Uncertainty and incomplete information: real environments are only partially known.
• Interpretability: deep models resist explanation, limiting trust in high-stakes use.
• Computational cost: training large models demands significant hardware and energy.
• Generalisation gap: laboratory accuracy often drops in deployment as conditions shift.