Unit 2: Linear time-invariant systems

I. Orientation: The LTI Framework

A system is a transformation mapping an input signal to an output signal, written y = T{x}. The power of the linear time-invariant (LTI) class is that a single function — the impulse response — completely characterizes the system, because any input can be decomposed into shifted, scaled impulses.

• Linearity: the system obeys superposition. If x₁ → y₁ and x₂ → y₂, then a·x₁ + b·x₂ → a·y₁ + b·y₂. Combines additivity and homogeneity (scaling).
• Time invariance: a shift in input produces the same shift in output. If x(t) → y(t), then x(t − t₀) → y(t − t₀).
• Impulse response: the output when the input is a unit impulse — h[n] = T{δ[n]} (discrete) or h(t) = T{δ(t)} (continuous). Every later section rests on this.
• Convention: discrete signals use bracket notation x[n], n ∈ ℤ; continuous signals use x(t), t ∈ ℝ.

II. Discrete-Time LTI Systems: The Convolution Sum

The convolution sum expresses any discrete LTI output as a weighted superposition of shifted impulse responses.

A. The convolution sum

• Sifting decomposition: any signal is x[n] = Σₖ x[k]·δ[n − k], a sum of scaled, shifted impulses.
• Superposition step: by time invariance δ[n − k] → h[n − k]; by linearity the scaled sum passes through:

y[n] = Σ_{k=-∞}^{∞} x[k]·h[n-k]  =  x[n] * h[n]

• Symbols: x[k] = input sample at index k; h[n − k] = impulse response shifted to k; * = convolution operator.
• Mechanics: flip h[k] to h[−k], shift by n, multiply point-by-point with x[k], and sum.

Worked example: x = [1, 2] (n=0,1), h = [1, 1, 1]. Overlap-and-add gives y = [1, 3, 3, 2] for n = 0…3; the output length is Nₓ + N_h − 1 = 4.

III. Continuous-Time and Discrete-Time Systems

Both domains share the same LTI machinery but differ in how signals are indexed and how the elementary building block behaves.

A. Continuous-time systems

• Signal model: x(t) defined for all real t; the elementary block is the continuous impulse δ(t) with ∫δ(t)dt = 1.
• Governing description: typically linear constant-coefficient differential equations.

B. Discrete-time systems

• Signal model: x[n] defined only at integers; the elementary block is the unit sample δ[n], which equals 1 at n = 0 and 0 elsewhere — a true, finite value (unlike δ(t)).
• Governing description: linear constant-coefficient difference equations.
• Contrast:
  1. Continuous: integration is the summation limit; δ(t) is a distribution of infinite height, zero width.
  2. Discrete: exact summation; δ[n] is an ordinary sequence, so computation is directly realizable.

IV. Basic System Properties

These properties classify any system, LTI or not, and translate into simple tests on h.

A. Basic system properties

• Memory / memoryless: output depends only on the present input if memoryless. Test: h[n] = K·δ[n] (discrete) or h(t) = K·δ(t).
• Causality: output depends on present and past inputs only. Test: h[n] = 0 for n < 0, h(t) = 0 for t < 0.
• Stability (BIBO): every bounded input yields a bounded output. Test: absolute summability/integrability —

Σ|h[n]| < ∞      ∫|h(t)|dt < ∞

• Invertibility: distinct inputs give distinct outputs; an inverse system h_inv exists with h * h_inv = δ.
• Time invariance & linearity: as defined in Section I; a system needs both to be LTI.

V. Continuous-Time LTI Systems: The Convolution Integral

The convolution integral is the continuous analogue of the sum, built from the sifting property of δ(t).

A. The convolution integral

• Sifting decomposition: x(t) = ∫_{-∞}^{∞} x(τ)·δ(t − τ) dτ.
• Superposition step: δ(t − τ) → h(t − τ); scaling by x(τ) and integrating gives:

y(t) = ∫_{-∞}^{∞} x(τ)·h(t-τ) dτ  =  x(t) * h(t)

• Symbols: τ = dummy integration variable; x(τ) = input weight; h(t − τ) = shifted, folded impulse response.
• Mechanics: reflect h(τ), slide by t, multiply overlap with x(τ), integrate.

Worked example: x(t) = h(t) = e^{-t}u(t). For t ≥ 0, y(t) = ∫₀ᵗ e^{-τ}e^{-(t-τ)}dτ = t·e^{-t}; zero for t < 0.

VI. Properties of Linear Time-Invariant Systems

Convolution inherits algebraic properties that let complex systems be simplified into single equivalents.

A. Properties of linear time invariant systems

• Commutative: x * h = h * x. Input and system roles can be swapped; output is unchanged.
• Associative: x * (h₁ * h₂) = (x * h₁) * h₂. A cascade of two LTI systems is one system with h = h₁ * h₂.
• Distributive: x * (h₁ + h₂) = x*h₁ + x*h₂. A parallel connection sums to h = h₁ + h₂.
• Identity: x * δ = x. The unit impulse is the convolution identity element.
• Property-to-h mapping: memoryless, causal, stable, and invertible all reduce to the tests in Section IV, so the impulse response encodes every structural property.

VII. Causal LTI Systems Described by Differential and Difference Equations

Practical LTI systems are rarely specified by h directly; they are given as equations relating input and output derivatives (or delays).

A. Causal LTI systems described by differential and difference equations

The general forms are:

Continuous:  Σ_{k=0}^{N} a_k · d^k y/dt^k = Σ_{k=0}^{M} b_k · d^k x/dt^k
Discrete:    Σ_{k=0}^{N} a_k · y[n-k]      = Σ_{k=0}^{M} b_k · x[n-k]

• Symbols: a_k, b_k = constant coefficients; N = order (highest derivative or largest delay); M = highest input term.
• Solution structure: total response = homogeneous (natural) response + particular (forced) response.
  • Homogeneous: solves the equation with x = 0; roots of the characteristic equation set the natural modes (e^{s_k t} or r_k^n).
  • Particular: matches the form of the driving input.
• Initial rest condition: for the system to be LTI and causal, assume the output and its derivatives are zero before the input begins — this fixes the arbitrary constants.
  1. Differential (continuous): recursion is implicit; solved via characteristic roots or Laplace methods.
  2. Difference (discrete): naturally recursive —

y[n] = (1/a_0)·( Σ b_k·x[n-k] − Σ_{k≥1} a_k·y[n-k] )

each output computed from past outputs and current/past inputs, ideal for direct programming.

VIII. Software Simulation of Convolution and Correlation

Simulation replaces analytic convolution with finite-length numerical summation, and extends the same machinery to correlation for similarity measurement.

A. Software simulation of convolution and correlation

• Convolution vs correlation:
  1. Convolution: folds one sequence — y[n] = Σ x[k]h[n−k]; measures system output.
  2. Correlation: no fold — r_xy[n] = Σ x[k]·y[k−n] (cross) or x = y (auto); measures similarity/lag. Correlation of x with a folded h equals convolution.
• Length rule: linear convolution of lengths Nₓ and N_h yields Nₓ + N_h − 1 samples.
• Library calls: numpy.convolve, scipy.signal.convolve, numpy.correlate in Python; conv and xcorr in MATLAB.

Reference pseudocode (direct convolution):

def convolve(x, h):
    Lx, Lh = len(x), len(h)
    y = [0.0] * (Lx + Lh - 1)     # output length Nx + Nh - 1
    for n in range(len(y)):
        for k in range(Lx):
            j = n - k              # index into h
            if 0 <= j < Lh:
                y[n] += x[k] * h[j]
    return y

• Symbols: Lx, Lh = input lengths; n = output index; k, j = overlap indices bounded to valid ranges.
• Cross-correlation edit: replace j = n - k with a lag-shifted index and remove the fold — reuse the same double loop, confirming the two operations differ only by reflection of one argument.