Unit 2 - Notes

ECE220

Unit 2: Linear Time-Invariant (LTI) Systems

1. Introduction

Signals and systems analysis relies heavily on the concept of Linear Time-Invariant (LTI) systems. While few physical systems are perfectly linear and time-invariant, many can be modeled as such to a high degree of accuracy. The power of LTI systems lies in the fact that they can be completely characterized by their response to a single specific signal: the impulse.

If we know the Impulse Response of an LTI system, we can calculate the output for any arbitrary input signal using Convolution.

Key Definitions

Linearity: A system is linear if it satisfies the Principle of Superposition, which includes:
- Additivity: Response to $x_1 + x_2$ is $y_1 + y_2$ .
- Homogeneity (Scaling): Response to $ax_1$ is $ay_1$ .
Time-Invariance: A time shift in the input signal results in an identical time shift in the output signal.
- If $x(t) \to y(t)$ , then $x(t-t_0) \to y(t-t_0)$ .

2. Continuous-Time and Discrete-Time Systems

Systems are classified based on the nature of the signals they process.

Continuous-Time (CT) Systems

Input/Output: Continuous variables defined at all instants of time $t$ .
Mathematical Model: Typically described by differential equations.
Notation: Parentheses, e.g., $x(t)$ , $y(t)$ .
Examples: Analog circuits (RC, RLC), mechanical dampers.

Discrete-Time (DT) Systems

Input/Output: Discrete sequences defined only at integer time steps $n$ .
Mathematical Model: Typically described by difference equations.
Notation: Square brackets, e.g., $x[n]$ , $y[n]$ .
Examples: Digital filters, computer algorithms, stock market data processing.

3. Discrete-Time LTI Systems: The Convolution Sum

Representation of Discrete Signals

Any discrete-time signal $x[n]$ can be represented as a sum of scaled, shifted unit impulses ( $\delta[n]$ ):

x[n] = \sum_{k=-\infty}^{\infty} x[k]\delta[n-k]

Here,

x[k]

acts as the coefficient (weight) for the impulse located at

n=k

Derivation of the Convolution Sum

Let $h[n]$ be the response of a linear system to a unit impulse $\delta[n]$ . If the system is Time-Invariant, the response to a shifted impulse $\delta[n-k]$ is $h[n-k]$ .

Applying Linearity (Superposition):

Input: $x[n] = \sum_{k=-\infty}^{\infty} x[k]\delta[n-k]$
Output $y[n]$ is the sum of the responses to the individual shifted impulses:
$y[n] = \sum_{k=-\infty}^{\infty} x[k]h[n-k]$

This formula is known as the Convolution Sum or discrete convolution. It is denoted as:

y[n] = x[n] * h[n]

Steps to Compute Convolution

To calculate $y[n]$ at a specific time $n$ :

Change of Variable: Rename the independent variable of the signals to $k$ (e.g., $x[k]$ , $h[k]$ ).
Fold (Reflect): Time-reverse the impulse response to get $h[-k]$ .
Shift: Shift the folded signal by $n$ to get $h[n-k]$ .
Multiply: Multiply $x[k]$ by $h[n-k]$ for all $k$ .
Sum: Sum the products over all $k$ to get the value of $y[n]$ .

4. Continuous-Time LTI Systems: The Convolution Integral

Representation of Continuous Signals

Analogous to the discrete case, a continuous signal $x(t)$ can be viewed as a sum of weighted impulses in the limit as the width of the impulses approaches zero.

x(t) = \int_{-\infty}^{\infty} x(\tau)\delta(t-\tau) d\tau

Derivation of the Convolution Integral

Let $h(t)$ be the impulse response of the CT LTI system (response to $\delta(t)$ ).
Using the same logic of Linearity and Time-Invariance:

The response to an impulse shifted to $\tau$ , $\delta(t-\tau)$ , is $h(t-\tau)$ .
The response to the weighted impulse $x(\tau)\delta(t-\tau)d\tau$ is $x(\tau)h(t-\tau)d\tau$ .
Integrating over all $\tau$ :

y(t) = \int_{-\infty}^{\infty} x(\tau)h(t-\tau) d\tau

This is the Convolution Integral, denoted as $y(t) = x(t) * h(t)$ .

Graphical Interpretation

Plot $x(\tau)$ and $h(\tau)$ on the $\tau$ axis.
Fold $h(\tau)$ about the vertical axis to get $h(-\tau)$ .
Shift $h(-\tau)$ by $t$ to the right (for $t>0$ ) or left (for $t<0$ ) to get $h(t-\tau)$ .
Multiply the two signals to find the overlap area.
Integrate the area under the product curve. This area equals $y(t)$ .

5. Basic System Properties (General)

Before discussing properties specific to LTI systems, we recall general system definitions:

Memory: A system has memory if the output at time $t$ depends on past or future inputs (not just the current input).
Invertibility: A system is invertible if distinct inputs produce distinct outputs (i.e., the input can be recovered from the output).
Causality: The output depends only on present and past inputs.
Stability: Bounded Input produces Bounded Output (BIBO).

6. Properties of Linear Time-Invariant Systems

For LTI systems, these properties can be defined entirely in terms of the impulse response $h$ .

A. Algebraic Properties of Convolution

Commutative Property:
$x(t) * h(t) = h(t) * x(t)$
Significance: The roles of input and impulse response are interchangeable.
Distributive Property:
$x * (h_1 + h_2) = x * h_1 + x * h_2$
Significance: Parallel connection of two systems ( $h_1$ and $h_2$ ) is equivalent to a single system with impulse response $h_1 + h_2$ .
Associative Property:
$x * (h_1 * h_2) = (x * h_1) * h_2$
Significance: Series (cascade) connection of systems allows the impulse responses to be convolved first. The order of cascaded LTI systems does not matter.

B. System Characteristics in Terms of Impulse Response

Memoryless:
- An LTI system is memoryless if $h[n] = K\delta[n]$ (Discrete) or $h(t) = K\delta(t)$ (Continuous).
- The output is simply a scaled version of the current input.
Causality:
- An LTI system is causal if $h(t) = 0$ for $t < 0$ (or $h[n] = 0$ for $n < 0$ ).
- This ensures the system does not respond before the input is applied.
Stability (BIBO):
- Discrete-Time: Stable if the impulse response is absolutely summable.
  $\sum_{k=-\infty}^{\infty} |h[k]| < \infty$
- Continuous-Time: Stable if the impulse response is absolutely integrable.
  $\int_{-\infty}^{\infty} |h(\tau)| d\tau < \infty$
Unit Step Response:
- The step response $s(t)$ is the running integral (CT) or running sum (DT) of the impulse response.
- $s(t) = \int_{-\infty}^{t} h(\tau) d\tau$
- $s[n] = \sum_{k=-\infty}^{n} h[k]$

7. Causal LTI Systems Described by Differential and Difference Equations

While convolution describes the input-output relationship explicitly, systems are often physically modeled using equations relating the rates of change (CT) or time differences (DT).

Linear Constant-Coefficient Differential Equations (LCCDE)

Used for Continuous-Time systems.

\sum_{k=0}^{N} a_k \frac{d^k y(t)}{dt^k} = \sum_{k=0}^{M} b_k \frac{d^k x(t)}{dt^k}

Complete Solution:
- $y_p(t)$ : Particular solution (forced response due to input).
- $y_h(t)$ : Homogeneous solution (natural response due to system structure).
Initial Conditions: Required to determine the auxiliary constants in the homogeneous solution. For causal LTI systems, we assume Initial Rest (if input is 0 for $t<0$ , output is 0 for $t<0$ ).

Linear Constant-Coefficient Difference Equations

Used for Discrete-Time systems.

\sum_{k=0}^{N} a_k y[n-k] = \sum_{k=0}^{M} b_k x[n-k]

Recursive Form:
- The output depends on previous outputs.
- Corresponds to IIR (Infinite Impulse Response) systems.
Non-Recursive Form: If (no feedback), depends only on current and past inputs.
- Corresponds to FIR (Finite Impulse Response) systems.

8. Software Simulation of Convolution and Correlation

Convolution Simulation

Software tools (MATLAB, Python/SciPy) perform discrete convolution numerically. Even for continuous signals, simulation requires discretization (sampling).

Concept:
Software creates an array for signal $x$ and signal $h$ . It implements the "sliding window" sum-of-products algorithm.

Length of output signal: If len( $x$ ) = $L$ and len( $h$ ) = $M$ , then len( $y$ ) = $L + M - 1$ .

Python Implementation:

PYTHON

import numpy as np

# Define signals
x = np.array([1, 2, 3])
h = np.array([1, 1])

# Built-in function
y = np.convolve(x, h)
# Result: [1, 3, 5, 3]

Correlation Simulation

Correlation measures the similarity between two signals as a function of the displacement of one relative to the other.

Cross-Correlation ( $r_{xy}$ ): Measure of similarity between two different signals and .
- Formula (DT): $r_{xy}[l] = \sum_{n=-\infty}^{\infty} x[n]y[n-l]$
- Difference from Convolution: Convolution involves folding ( $y[n-k]$ ), Correlation does not ( $y[n-l]$ ).
- Relation: $r_{xy}[n] = x[n] * y[-n]$ .
Auto-Correlation ( $r_{xx}$ ): Correlation of a signal with itself. Used for finding repeating patterns or signal periodicity in noise. maximum value always occurs at lag $l=0$ .