Back to Subject

Unit 1: Matrix methods and linear systems

MTH165 — Mathematics For Engineers 5 min read

I. Orientation — Matrices as representations of linear transformations

A matrix is a rectangular array of scalars used to represent linear equations, vectors, and linear transformations. Matrix methods convert coupled engineering equations into forms that can be systematically reduced, solved, and analysed.

  • Underlying field: Entries usually belong to (\mathbb R) or (\mathbb C), denoted by (\mathbb F).
  • Order convention: A matrix (A=[a{ij}]) with (m) rows and (n) columns has order (m\times n); (a{ij}) is its entry in row (i), column (j).
  • Vector convention: Vectors are normally column matrices; (x\in\mathbb F^n) means that (x) has (n) components.
  • Linear-system model: A system of (m) equations in (n) unknowns is represented as (Ax=b), where (A\in\mathbb F^{m\times n}).
  • Transformation viewpoint: An (m\times n) matrix defines the linear map (T(x)=Ax) from (\mathbb F^n) to (\mathbb F^m).
  • Central principle: Row operations simplify equations without changing their solution set, while similarity transformations preserve the eigenvalue structure of a square matrix.

II. Matrices — Structure and algebra

A. Review of matrices

A matrix is characterised by its dimensions, entries, and algebraic properties.

  • Common types:
    • Row and column matrices: Orders (1\times n) and (m\times1), respectively.
    • Square matrix: (A\in\mathbb F^{n\times n}).
    • Diagonal matrix: (a_{ij}=0) whenever (i\ne j).
    • Identity matrix: (I_n=\operatorname{diag}(1,\ldots,1)), satisfying (AI_n=I_nA=A).
    • Symmetric matrix: (A^T=A); a complex Hermitian matrix satisfies (A^*=A).
  • Equality: (A=B) only when both matrices have the same order and (a{ij}=b{ij}) for every (i,j).
  • Addition and scalar multiplication: For equal-sized matrices,
    TEXT
    (A+B)ij = aij + bij,       (cA)ij = c aij

    Here (c) is a scalar.
  • Matrix multiplication: If (A) is (m\times n) and (B) is (n\times p), then (AB) is (m\times p):
    TEXT
    (AB)ij = Σ(k=1 to n) aik bkj

    Multiplication is associative and distributive but generally (AB\ne BA).
  • Transpose laws:
    TEXT
    (A+B)T = AT + BT,          (AB)T = BT AT
  • Determinant: For square (A), (\det(A)) measures singularity; (A) is nonsingular exactly when (\det(A)\ne0).

III. Matrix Reduction — Equivalent forms

A. Elementary operations of matrices

Elementary operations simplify a matrix while preserving the essential equivalence of the associated equations.

  • Elementary row operations:
    1. Interchange rows: (R_i\leftrightarrow R_j).
    2. Scale a row: (R_i\leftarrow cR_i), where (c\ne0).
    3. Replace a row: (R_i\leftarrow R_i+cR_j), where (i\ne j).
  • Elementary matrices: Applying one row operation to (I_m) produces an elementary matrix (E); the same operation on (A) is represented by (EA).
  • Reversibility: Every elementary operation has an inverse, so every elementary matrix is nonsingular.
  • Row-echelon form: All zero rows lie below nonzero rows, each leading entry lies to the right of the one above it, and entries below each pivot are zero.
  • Reduced row-echelon form: Each pivot equals (1) and is the only nonzero entry in its column; this form is unique.
  • Column operations: Analogous operations act on columns through right multiplication, (AE), but they generally change the variables in (Ax=b) and must not be used casually on an augmented system.

IV. Rank — Number of independent directions

A. Rank of a matrix

The rank of (A), written (\operatorname{rank}(A)), is the dimension of its row space or, equivalently, its column space.

  • Equivalent descriptions: Rank equals the number of pivots, the number of nonzero rows in echelon form, and the order of the largest nonzero minor.
  • Bounds: If (A) is (m\times n),
    TEXT
    0 ≤ rank(A) ≤ min(m,n)
  • Full rank: (A) has full row rank if (\operatorname{rank}(A)=m), and full column rank if (\operatorname{rank}(A)=n).
  • Invariance: Elementary row and column operations do not change rank.
  • Useful inequalities:
    TEXT
    rank(AB) ≤ min(rank(A), rank(B))
rank(A+B) ≤ rank(A) + rank(B)
  • Worked example: For
    TEXT
    A = [1  2  3
     2  4  6
     1  1  1]

    applying (R_2\leftarrow R_2-2R_1) gives one zero row, while the other two rows are independent. Hence (\operatorname{rank}(A)=2).

V. Vector Relations — Span and basis behaviour

A. Linear dependence and independence of vectors

Vectors (v_1,\ldots,v_k) are linearly independent when no nontrivial linear combination of them equals the zero vector.

  • Defining equation:
    TEXT
    c1v1 + c2v2 + ··· + ckvk = 0

    Independence requires (c_1=\cdots=c_k=0); otherwise the vectors are dependent.
  • Matrix test: Form (A=[v_1\ v_2\ \cdots\ v_k]). The vectors are independent exactly when (\operatorname{rank}(A)=k).
  • Homogeneous-system test: Independence is equivalent to (Ac=0) having only the trivial solution (c=0).
  • Immediate dependence: A set is dependent if it contains the zero vector, repeats a vector, or has more than (n) vectors in (\mathbb F^n).
  • Span: The set of all combinations (\sum c_iv_i) is (\operatorname{span}{v_1,\ldots,v_k}).
  • Basis: A basis is both independent and spanning; every vector in the space then has unique basis coordinates.

VI. Linear Systems — Existence and computation

A. Solution of linear system of equations

A linear system (Ax=b) is solved by reducing its augmented matrix ([A\mid b]) to echelon or reduced row-echelon form.

  • Consistency criterion: The Rouché–Capelli theorem states
    TEXT
    Ax = b is consistent ⇔ rank(A) = rank([A|b]).
  • Unique solution:
    TEXT
    rank(A) = rank([A|b]) = n

    where (n) is the number of unknowns.
  • Infinitely many solutions: If the common rank is less than (n), there are (n-\operatorname{rank}(A)) free variables.
  • No solution: If (\operatorname{rank}(A)<\operatorname{rank}([A\mid b])), reduction produces a contradictory row such as (0=1).
  • Homogeneous system: (Ax=0) is always consistent. It has a nonzero solution precisely when (\operatorname{rank}(A)<n).
  • General consistent solution: If (x_p) is one particular solution, then
    TEXT
    x = xp + xn

    where (x_n) is any vector in the null space (\mathcal N(A)={x:Ax=0}).

VII. Matrix Inversion — Reversing a transformation

A. Inverse of matrices

The inverse of a square matrix (A) is the matrix (A^{-1}) satisfying (AA^{-1}=A^{-1}A=I).

  • Existence conditions: For (A\in\mathbb F^{n\times n}), the following are equivalent:
    • (A^{-1}) exists.
    • (\det(A)\ne0).
    • (\operatorname{rank}(A)=n).
    • (Ax=0) has only (x=0).
  • Adjugate formula:
    TEXT
    A⁻¹ = adj(A) / det(A),       det(A) ≠ 0

    Here (\operatorname{adj}(A)) is the transpose of the cofactor matrix.
  • Gauss–Jordan method: Row-reduce the block matrix
    TEXT
    [A | I]  →  [I | A⁻¹].
  • Algebraic laws:
    TEXT
    (AB)⁻¹ = B⁻¹A⁻¹,     (AT)⁻¹ = (A⁻¹)T
  • Solving systems: If (A) is nonsingular, (Ax=b) has the unique solution (x=A^{-1}b); computational methods usually use factorisation rather than explicitly forming (A^{-1}).

VIII. Spectral Analysis — Invariant directions

A. Eigenvalues and eigenvectors

A nonzero vector (v) is an eigenvector of a square matrix (A) if multiplication by (A) only scales it.

  • Eigenvalue equation:
    TEXT
    Av = λv,       v ≠ 0

    Here (\lambda) is the corresponding eigenvalue.
  • Characteristic equation:
    TEXT
    det(A − λI) = 0

    Its roots are the eigenvalues of (A).
  • Eigenspace: For eigenvalue (\lambda),
    TEXT
    Eλ = null(A − λI).

    Any nonzero member of (E_\lambda) is an eigenvector.
  • Geometric meaning: Eigenvectors identify directions preserved by the transformation; (|\lambda|) gives the scaling magnitude, while a negative real (\lambda) reverses direction.
  • Diagonalisation: If (A) has (n) independent eigenvectors (v_i), then
    TEXT
    A = PDP⁻¹

    where (P=[v_1\ \cdots\ v_n]) and (D=\operatorname{diag}(\lambda_1,\ldots,\lambda_n)).

IX. Spectral Rules — Consequences of the characteristic polynomial

A. Properties of eigenvalues

Eigenvalues connect matrix operations with scalar quantities such as determinant, trace, and powers.

  • Trace and determinant: Counting algebraic multiplicity,
    TEXT
    Σ λi = tr(A),        Π λi = det(A).
  • Triangular matrices: The eigenvalues of a triangular or diagonal matrix are its diagonal entries.
  • Singularity: (0) is an eigenvalue exactly when (\det(A)=0).
  • Similarity: If (B=P^{-1}AP), then (A) and (B) have the same characteristic polynomial and eigenvalues.
  • Transposition: (A) and (A^T) have the same eigenvalues because
    TEXT
    det(AT − λI) = det(A − λI).
  • Spectral mapping: If (Av=\lambda v), then (A^kv=\lambda^kv) and (p(A)v=p(\lambda)v) for any polynomial (p).
  • Shift and inverse: Eigenvalues of (A+cI) are (\lambda_i+c); when (A) is invertible, eigenvalues of (A^{-1}) are (1/\lambda_i).
  • Multiplicity: Geometric multiplicity (\dim E_\lambda) is at least (1) and no greater than algebraic multiplicity.

X. Polynomial Identity — Matrix reduction by its characteristic equation

A. Cayley-Hamilton theorem

The Cayley–Hamilton theorem states that every square matrix satisfies its own characteristic equation.

  • Formal statement: If
    TEXT
    p(λ) = det(λI − A)
     = λⁿ + c(n−1)λⁿ⁻¹ + ··· + c1λ + c0,

    then substitution of (A) gives
    TEXT
    p(A) = Aⁿ + c(n−1)Aⁿ⁻¹ + ··· + c1A + c0I = 0.
  • Meaning of substitution: The scalar constant becomes (c_0I), ensuring that every term is an (n\times n) matrix.
  • Reduction of powers: The identity expresses (A^n), and therefore every higher power, as a combination of (I,A,\ldots,A^{n-1}).
  • Finding an inverse: If (c_0\ne0), then (A) is invertible and
    TEXT
    A⁻¹ = −(1/c0)[Aⁿ⁻¹ + c(n−1)Aⁿ⁻² + ··· + c1I].
  • Worked example: For (A=\begin{bmatrix}1&1\0&2\end{bmatrix}),
    TEXT
    p(λ) = (λ−1)(λ−2) = λ²−3λ+2.

    Hence (A^2-3A+2I=0). Multiplying by (A^{-1}) gives (A-3I+2A^{-1}=0), so
    TEXT
    A⁻¹ = (3I−A)/2 = [1  −1/2
                   0   1/2].
Next
Unit 2