Unit 2 - Notes

ECE220 14 min read

Unit 2: Linear time-invariant systems

1. Introduction

Linear Time-Invariant (LTI) systems are a fundamental concept in signals and systems. They are a class of systems that are both linear and time-invariant. The significance of LTI systems lies in their analytical tractability. The output of an LTI system for any given input can be determined if we know its response to a single, simple input: the unit impulse. This response is called the impulse response.

The entire behavior of an LTI system is completely characterized by its impulse response. The mathematical operation that relates the input, the impulse response, and the output is convolution.

For Discrete-Time (DT) systems, this operation is the convolution sum.
For Continuous-Time (CT) systems, it is the convolution integral.

Understanding LTI systems allows us to design filters, analyze communication channels, and model a vast array of physical phenomena.

2. Basic System Properties

Before delving into LTI systems, we must define the properties that constitute them. Let a system be represented by an operator $T\{\cdot\}$ such that the output is $y(t) = T\{x(t)\}$ for CT systems, or $y[n] = T\{x[n]\}$ for DT systems.

2.1. Linearity

A system is linear if it satisfies the properties of homogeneity (scaling) and additivity. Combined, this is known as the superposition principle.

Additivity: $T\{x_1(t) + x_2(t)\} = T\{x_1(t)\} + T\{x_2(t)\}$
Homogeneity: $T\{a \cdot x(t)\} = a \cdot T\{x(t)\}$ , for any constant $a$ .

Combined Linearity Condition:
A system is linear if and only if for any two inputs $x_1(t)$ and $x_2(t)$ and any constants $a_1$ and $a_2$ :

$T\{a_1 x_1(t) + a_2 x_2(t)\} = a_1 T\{x_1(t)\} + a_2 T\{x_2(t)\}$

The same principle applies to discrete-time systems with $t$ replaced by $n$ .

Example: $y(t) = 3x(t)$ is linear. $y(t) = x^2(t)$ is not linear.

2.2. Time-Invariance

A system is time-invariant if a time shift in the input signal causes an identical time shift in the output signal. The characteristics of the system do not change over time.

Continuous-Time: If $y(t) = T\{x(t)\}$ , then the system is time-invariant if $y(t-t_0) = T\{x(t-t_0)\}$ for any $t_0$ .
Discrete-Time: If $y[n] = T\{x[n]\}$ , then the system is time-invariant if $y[n-n_0] = T\{x[n-n_0]\}$ for any $n_0$ .

Example: $y(t) = x(t) + x(t-1)$ is time-invariant. $y(t) = t \cdot x(t)$ is time-variant because the system's behavior (the scaling factor $t$ ) depends on time.

2.3. Causality

A system is causal if its output at any given time depends only on the present and/or past values of the input. It does not depend on future values of the input.

CT Causal System: The output $y(t_0)$ depends only on the input $x(t)$ for $t \le t_0$ .
DT Causal System: The output $y[n_0]$ depends only on the input $x[n]$ for $n \le n_0$ .

All real-time physical systems are causal.

Example: $y(t) = x(t-2)$ is causal. $y(t) = x(t+2)$ is non-causal.

2.4. Stability (BIBO Stability)

A system is Bounded-Input, Bounded-Output (BIBO) stable if every bounded input produces a bounded output.

Bounded Input: There exists a finite constant $M_x$ such that $|x(t)| \le M_x < \infty$ for all $t$ (or $|x[n]| \le M_x < \infty$ for all $n$ ).
Bounded Output: There exists a finite constant $M_y$ such that $|y(t)| \le M_y < \infty$ for all $t$ (or $|y[n]| \le M_y < \infty$ for all $n$ ).

Example: $y(t) = x^2(t)$ is stable. An integrator, $y(t) = \int_{-\infty}^{t} x(\tau) d\tau$ , is unstable (e.g., a constant input $x(t)=1$ produces an unbounded ramp output $y(t)=t$ ).

2.5. Memory

Memoryless (or Static) System: A system is memoryless if its output at any time depends only on the input at that same time.
- Example: $y[n] = (x[n])^2 - 2x[n]$
System with Memory (or Dynamic): A system has memory if its output depends on past or future values of the input.
- Example: $y[n] = x[n] + x[n-1]$

3. Discrete-Time LTI Systems: The Convolution Sum

For a DT LTI system, if we know the impulse response $h[n]$ , which is the system's output to a unit impulse input $\delta[n]$ , we can find the output $y[n]$ for any input $x[n]$ .

3.1. The Sifting Property of the Unit Impulse

Any discrete-time signal $x[n]$ can be represented as a sum of scaled and shifted unit impulses:

$x[n] = \sum_{k=-\infty}^{\infty} x[k] \delta[n-k]$

This is called the sifting property. It breaks down the signal $x[n]$ into a series of simple building blocks (impulses).

3.2. Derivation of the Convolution Sum

Let's find the output $y[n]$ of an LTI system with input $x[n]$ .

Represent the input using the sifting property: $x[n] = \sum_{k=-\infty}^{\infty} x[k] \delta[n-k]$ .
The output is $y[n] = T\{x[n]\} = T\{\sum_{k=-\infty}^{\infty} x[k] \delta[n-k]\}$ .
Using the linearity property (additivity and scaling), we can move the operator $T$ inside the summation:
$y[n] = \sum_{k=-\infty}^{\infty} x[k] T\{\delta[n-k]\}$ .
The impulse response is defined as $h[n] = T\{\delta[n]\}$ . Using the time-invariance property, we know that $T\{\delta[n-k]\} = h[n-k]$ .
Substituting this back, we get the Convolution Sum:

$y[n] = \sum_{k=-\infty}^{\infty} x[k] h[n-k]$

This operation is denoted by an asterisk: $y[n] = x[n] * h[n]$ .

3.3. Mechanics of Convolution Sum

To compute $y[n]$ for a specific value of $n$ :

Index: Choose a value for $n$ .
Flip: Take the impulse response $h[k]$ and flip it about the vertical axis to get $h[-k]$ .
Shift: Shift the flipped sequence $h[-k]$ by $n$ to get $h[n-k]$ . If $n > 0$ , shift right. If $n < 0$ , shift left.
Multiply: Multiply the input signal $x[k]$ by the shifted, flipped impulse response $h[n-k]$ for all values of $k$ .
Sum: Sum the products from the previous step over all values of $k$ . The result is the single value $y[n]$ .
Repeat for all desired values of $n$ .

Example:
Let $x[n] = \{1, 2, 1\}$ and $h[n] = \{1, 1, 1\}$ .
The convolution $y[n] = x[n] * h[n]$ is:

$y[0] = \sum_k x[k]h[0-k] = x[0]h[0] + x[1]h[-1] + ... = (1)(1) = 1$
$y[1] = \sum_k x[k]h[1-k] = x[0]h[1] + x[1]h[0] + x[2]h[-1] = (1)(1) + (2)(1) = 3$
$y[2] = \sum_k x[k]h[2-k] = x[0]h[2] + x[1]h[1] + x[2]h[0] = (1)(1) + (2)(1) + (1)(1) = 4$
$y[3] = \sum_k x[k]h[3-k] = x[1]h[2] + x[2]h[1] = (2)(1) + (1)(1) = 3$
$y[4] = \sum_k x[k]h[4-k] = x[2]h[2] = (1)(1) = 1$
So, $y[n] = \{1, 3, 4, 3, 1\}$ .

4. Continuous-Time LTI Systems: The Convolution Integral

The concept for CT LTI systems is analogous to the DT case, with sums replaced by integrals and the Kronecker delta $\delta[n]$ replaced by the Dirac delta function $\delta(t)$ .

4.1. The Sifting Property of the Dirac Delta

Any continuous-time signal $x(t)$ can be represented as an integral of scaled and shifted Dirac delta functions:

$x(t) = \int_{-\infty}^{\infty} x(\tau) \delta(t-\tau) d\tau$

4.2. Derivation of the Convolution Integral

The derivation follows the same logic as the discrete case:

Represent the input: $x(t) = \int_{-\infty}^{\infty} x(\tau) \delta(t-\tau) d\tau$ .
Output is $y(t) = T\{x(t)\} = T\{\int_{-\infty}^{\infty} x(\tau) \delta(t-\tau) d\tau\}$ .
Using linearity (the integral is a limit of a sum):
$y(t) = \int_{-\infty}^{\infty} x(\tau) T\{\delta(t-\tau)\} d\tau$ .
The impulse response is $h(t) = T\{\delta(t)\}$ . By time-invariance, $T\{\delta(t-\tau)\} = h(t-\tau)$ .
Substituting this gives the Convolution Integral:

$y(t) = \int_{-\infty}^{\infty} x(\tau) h(t-\tau) d\tau$

This is denoted as $y(t) = x(t) * h(t)$ .

4.3. Mechanics of Convolution Integral (Graphical Method)

To compute $y(t)$ :

Index: Choose a value for $t$ .
Flip: Flip the impulse response $h(\tau)$ about the vertical axis to get $h(-\tau)$ .
Shift: Shift $h(-\tau)$ by $t$ to obtain $h(t-\tau)$ . If $t > 0$ , shift right. If $t < 0$ , shift left.
Multiply: Multiply the two signals, $x(\tau)$ and $h(t-\tau)$ .
Integrate: Calculate the area under the product curve from the previous step. This area is the value of $y(t)$ for the chosen $t$ .
Repeat for all $t$ to trace out the full output signal $y(t)$ .

5. Properties of Linear Time-Invariant Systems

The convolution operation has several important algebraic properties that reflect the properties of LTI systems.

5.1. Commutative Property

The order of convolution does not matter.

DT: $x[n] * h[n] = h[n] * x[n]$
CT: $x(t) * h(t) = h(t) * x(t)$
Implication: The roles of input and impulse response are interchangeable.

5.2. Associative Property

DT: $(x[n] * h_1[n]) * h_2[n] = x[n] * (h_1[n] * h_2[n])$
CT: $(x(t) * h_1(t)) * h_2(t) = x(t) * (h_1(t) * h_2(t))$
Implication: For systems connected in cascade (series), the overall impulse response is the convolution of the individual impulse responses. The order in which they are cascaded does not matter.

5.3. Distributive Property

DT: $x[n] * (h_1[n] + h_2[n]) = (x[n] * h_1[n]) + (x[n] * h_2[n])$
CT: $x(t) * (h_1(t) + h_2(t)) = (x(t) * h_1(t)) + (x(t) * h_2(t))$
Implication: For systems connected in parallel, the overall impulse response is the sum of the individual impulse responses.

5.4. LTI System Properties and the Impulse Response

The general properties of a system (causality, stability, etc.) can be determined directly from its impulse response $h$ .

Causality

An LTI system is causal if and only if its impulse response is zero for negative time.

DT: $h[n] = 0$ for $n < 0$ .
CT: $h(t) = 0$ for $t < 0$ .
This makes intuitive sense: the system cannot respond to an impulse before the impulse occurs at $t=0$ or $n=0$ .

Stability (BIBO)

An LTI system is BIBO stable if and only if its impulse response is absolutely summable (DT) or absolutely integrable (CT).

DT: The system is stable if $\sum_{n=-\infty}^{\infty} |h[n]| < \infty$ .
CT: The system is stable if $\int_{-\infty}^{\infty} |h(\tau)| d\tau < \infty$ .

Memory

An LTI system is memoryless if and only if its impulse response is a scaled impulse.

DT: $h[n] = K \delta[n]$ for some constant $K$ . The output is $y[n] = K x[n]$ .
CT: $h(t) = K \delta(t)$ for some constant $K$ . The output is $y(t) = K x(t)$ .
If $h$ is non-zero for any other time, the system has memory.

6. Causal LTI Systems Described by Differential and Difference Equations

Many physical systems are described by linear constant-coefficient differential or difference equations. These are mathematical representations of LTI systems.

6.1. Continuous-Time: Differential Equations

A general Nth-order linear constant-coefficient differential equation is:

$\sum_{k=0}^{N} a_k \frac{d^k y(t)}{dt^k} = \sum_{k=0}^{M} b_k \frac{d^k x(t)}{dt^k}$

The coefficients $a_k$ and $b_k$ are constants.
The "linear constant-coefficient" nature ensures the system is LTI.
To ensure causality, we must impose a condition of initial rest. This means that if the input $x(t) = 0$ for $t < t_0$ , then the output $y(t) = 0$ for $t < t_0$ . The system is "at rest" before the input is applied.

The solution $y(t)$ consists of two parts:

Homogeneous Solution ( $y_h(t)$ ): The solution to the equation with the input set to zero. It represents the natural response of the system.
Particular Solution ( $y_p(t)$ ): A solution that depends on the specific form of the input $x(t)$ . It represents the forced response of the system.

The total solution is $y(t) = y_h(t) + y_p(t)$ .

6.2. Discrete-Time: Difference Equations

A general Nth-order linear constant-coefficient difference equation is:

$\sum_{k=0}^{N} a_k y[n-k] = \sum_{k=0}^{M} b_k x[n-k]$

This equation can be rearranged to express $y[n]$ in terms of past values of the output and present/past values of the input, making it suitable for computation. If $a_0 \neq 0$ :

$y[n] = \frac{1}{a_0} \left( \sum_{k=0}^{M} b_k x[n-k] - \sum_{k=1}^{N} a_k y[n-k] \right)$
Recursive Form: If any $a_k \neq 0$ for $k \ge 1$ , the output depends on its own past values, and the system is called recursive. This corresponds to an Infinite Impulse Response (IIR) system.
Non-Recursive Form: If all $a_k = 0$ for $k \ge 1$ , the output depends only on the input. This corresponds to a Finite Impulse Response (FIR) system. The equation is then simply the convolution sum.
Like the continuous case, the system is LTI and causal under the condition of initial rest.

7. Software Simulation of Convolution and Correlation

Python, with libraries like NumPy, SciPy, and Matplotlib, is excellent for simulating these operations.

7.1. Convolution

Convolution finds the output of an LTI system given an input and the system's impulse response.

PYTHON

import numpy as np
import matplotlib.pyplot as plt

# Define the input signal x[n] and impulse response h[n]
# Example: a square pulse input and a decaying exponential impulse response
x = np.repeat([0., 1., 0.], 10)  # A square pulse
h = np.exp(-np.arange(20) * 0.3) # Decaying exponential

# Compute the convolution
y = np.convolve(x, h, mode='full')

# Create time axes for plotting
n_x = np.arange(len(x))
n_h = np.arange(len(h))
n_y = np.arange(len(y))

# Plotting
plt.figure(figsize=(12, 8))

plt.subplot(3, 1, 1)
plt.stem(n_x, x)
plt.title('Input Signal x[n]')
plt.grid(True)

plt.subplot(3, 1, 2)
plt.stem(n_h, h)
plt.title('Impulse Response h[n]')
plt.grid(True)

plt.subplot(3, 1, 3)
plt.stem(n_y, y)
plt.title('Output Signal y[n] = x[n] * h[n]')
plt.xlabel('n')
plt.grid(True)

plt.tight_layout()
plt.show()

The mode argument in np.convolve can be:

'full': Returns the full convolution product (length len(x) + len(h) - 1). This is the default.
'same': Returns an output of the same length as the longer input.
'valid': Returns only the parts of the convolution computed without zero-padding.

7.2. Correlation

Correlation is a very similar operation to convolution but is used to measure the similarity between two signals as a function of the time-lag applied to one of them.

Cross-correlation: Measures similarity between two different signals.
Autocorrelation: Measures similarity between a signal and a time-shifted version of itself.

The formula for cross-correlation is:

$r_{xy}[n] = \sum_{k=-\infty}^{\infty} x[k] y[n+k]$

Key Difference: In convolution, the second signal is flipped ( $h[n-k]$ ). In correlation, it is not ( $y[n+k]$ ). This means correlation is equivalent to convolving with a time-reversed signal: $r_{xy}[n] = x[n] * y[-n]$ .