Unit 1: Matrix methods and linear systems

I. Orientation — The Matrix Framework

Matrix methods organize numerical information and transform systems of linear equations into forms that can be solved systematically. Their governing principle is that permitted row operations preserve the solution set while revealing structural information such as rank, invertibility, and eigenvalues.

A. Defining framework

The unit rests on a small set of algebraic conventions used throughout matrix analysis.

• Scalar field: Matrix entries usually belong to the real numbers (\mathbb R) or complex numbers (\mathbb C).
• Order convention: An (m\times n) matrix has (m) rows and (n) columns; its entry in row (i), column (j) is (a_{ij}).
• Compatibility: Matrices can be added only when they have equal orders; (AB) is defined when the number of columns of (A) equals the number of rows of (B).
• Linear-system convention: A system is represented as (AX=B), where (A) is the coefficient matrix, (X) the column of unknowns, and (B) the constant column.
• Square-matrix structure: Determinants, ordinary inverses, eigenvalues, and the Cayley–Hamilton theorem concern square matrices (A\in\mathbb F^{n\times n}), where (\mathbb F) is the scalar field.

II. Matrices — Representation and Basic Algebra

A. review of matrices

A matrix is a rectangular array of scalars used to represent linear transformations, equation systems, and engineering data.

• General form: An (m\times n) matrix is written
  
  TEXT

    A = [aᵢⱼ],  1 ≤ i ≤ m,  1 ≤ j ≤ n

  
  Here (a_{ij}) is an entry, while (m) and (n) are the row and column counts.
• Important types:
  • Row and column matrices: Orders (1\times n) and (m\times1), respectively.
  • Square matrix: Has order (n\times n).
  • Diagonal matrix: Satisfies (a_{ij}=0) whenever (i\ne j).
  • Identity matrix: (I_n=\operatorname{diag}(1,\ldots,1)), with (AI_n=I_nA=A).
  • Symmetric matrix: Satisfies (A^T=A), where (A^T) denotes the transpose.
• Operations:
  
  TEXT

    (A + B)ᵢⱼ = aᵢⱼ + bᵢⱼ
    (kA)ᵢⱼ = kaᵢⱼ
    (AB)ᵢⱼ = Σᵣ aᵢᵣbᵣⱼ

  
  Here (k) is a scalar and (r) runs across the common inner dimension.
• Multiplication warning: Matrix multiplication is associative and distributive but generally not commutative: (AB\ne BA).

III. Elementary Operations — Systematic Matrix Reduction

A. elementary operations of matrices

Elementary operations reduce a matrix to echelon or canonical form without losing the essential information relevant to a linear system.

• Three row operations:
  1. Interchange: Swap rows (R_i\leftrightarrow R_j).
  2. Scaling: Replace (R_i) by (kR_i), where (k\ne0).
  3. Replacement: Replace (R_i) by (R_i+kR_j), where (i\ne j).
• Column operations: The corresponding operations apply to columns, although arbitrary column operations on an augmented system alter the meanings of its variables.
• Echelon forms:
  • Row-echelon form: Zero rows lie below nonzero rows, and each leading entry lies right of the one above it.
  • Reduced row-echelon form: Every leading entry is (1) and is the only nonzero entry in its column.
• Elementary matrices: Performing one row operation on (I_m) produces an elementary matrix (E); applying the operation to (A) is equivalent to forming (EA).
• Preserved quantities: Row operations preserve the solution set of an augmented system and preserve matrix rank.

IV. Rank — Measuring Independent Information

A. rank of a matrix

The rank of a matrix is the maximum number of linearly independent rows or columns, measuring how much independent information the matrix contains.

• Equivalent definitions: For (A), rank is:
  • the dimension of its row space;
  • the dimension of its column space;
  • the order of its largest nonzero minor.
• Row–column equality: Row rank always equals column rank, so both are denoted (\operatorname{rank}(A)).
• Reduction method: Reduce (A) to row-echelon form; the number of pivots or nonzero rows equals its rank.
• Bounds:
  
  TEXT

    0 ≤ rank(A) ≤ min(m,n)

  
  Here (A) has order (m\times n).
• Full rank: A square (n\times n) matrix has full rank when (\operatorname{rank}(A)=n); this is equivalent to (\det(A)\ne0).
• Product inequality:
  
  TEXT

    rank(AB) ≤ min(rank(A), rank(B))

  
  The product (AB) must be dimensionally defined.

V. Vector Dependence — Detecting Redundant Directions

A. linear dependence and independence of vectors

Vectors are linearly independent when none can be constructed as a linear combination of the others.

• Formal test: Vectors (v_1,\ldots,v_k) are independent if
  
  TEXT

    c₁v₁ + c₂v₂ + ··· + cₖvₖ = 0

  
  implies (c_1=c_2=\cdots=c_k=0), where the (c_i) are scalars and (0) is the zero vector.
• Dependence condition: The vectors are dependent if the same equation has a solution in which at least one (c_i\ne0).
• Matrix test: Place the vectors as columns of (A). They are independent exactly when every column is a pivot column, so (\operatorname{rank}(A)=k).
• Dimension rule: Any collection of more than (n) vectors in (\mathbb F^n) is dependent.
• Determinant test: Exactly (n) vectors in (\mathbb F^n) are independent when the determinant of the matrix formed from them is nonzero.
• Engineering meaning: Dependence reveals redundant equations, repeated measurements, or directions that add no new degree of freedom.

VI. Linear Systems — Existence and Classification of Solutions

A. solution of linear system of equations

A linear system is solved by reducing its augmented matrix and comparing the independent constraints with the number of unknowns.

• Matrix representation:
  
  TEXT

    AX = B,       [A | B]

  
  Here (A) is an (m\times n) coefficient matrix, (X) contains (n) unknowns, (B) contains (m) constants, and ([A\mid B]) is the augmented matrix.
• Consistency criterion: The Rouché–Capelli theorem states
  
  TEXT

    System is consistent ⇔ rank(A) = rank([A | B])

• Solution classification:
  1. Unique solution: (\operatorname{rank}(A)=\operatorname{rank}([A\mid B])=n).
  2. Infinitely many solutions: The two ranks are equal but less than (n); the number of free variables is (n-\operatorname{rank}(A)).
  3. No solution: (\operatorname{rank}(A)\ne\operatorname{rank}([A\mid B])).
• Homogeneous system: (AX=0) is always consistent. It has a nonzero solution exactly when (\operatorname{rank}(A)<n).
• Elimination method: Apply row operations to ([A\mid B]), identify pivots and free variables, then use back-substitution or read the solution from reduced row-echelon form.
• Square nonsingular case: If (\det(A)\ne0), the unique solution is (X=A^{-1}B).

VII. Matrix Inversion — Reversing a Linear Transformation

A. inverse of matrices

The inverse of a square matrix reverses its action and exists precisely when the matrix is nonsingular.

• Definition:
  
  TEXT

    AA⁻¹ = A⁻¹A = Iₙ

  
  Here (A^{-1}) is the inverse and (I_n) is the identity matrix of order (n).
• Existence conditions: For an (n\times n) matrix, the following are equivalent:
  • (\det(A)\ne0);
  • (\operatorname{rank}(A)=n);
  • the columns of (A) are independent;
  • (AX=B) has a unique solution for every (B).
• Gauss–Jordan method:
  
  TEXT

    [A | Iₙ]  →  [Iₙ | A⁻¹]

  
  Apply row operations until the left block becomes the identity.
• Adjugate formula:
  
  TEXT

    A⁻¹ = adj(A) / det(A)

  
  Here (\operatorname{adj}(A)) is the transpose of the cofactor matrix.
• Properties:
  
  TEXT

    (AB)⁻¹ = B⁻¹A⁻¹
    (Aᵀ)⁻¹ = (A⁻¹)ᵀ

  
  Both formulas require the relevant inverses to exist.

VIII. Eigenpairs — Invariant Directions and Scale Factors

A. eigenvalues and eigenvectors

An eigenvector is a nonzero direction whose orientation is unchanged by a linear transformation, while its eigenvalue gives the corresponding scale factor.

• Defining equation:
  
  TEXT

    Av = λv,    v ≠ 0

  
  Here (A) is square, (v) is an eigenvector, and (\lambda) is its eigenvalue.
• Characteristic equation:
  
  TEXT

    det(A − λIₙ) = 0

  
  This condition follows because ((A-\lambda I_n)v=0) must have a nonzero solution.
• Computational procedure:
  1. Find roots (\lambda) of the characteristic polynomial.
  2. For each root, solve ((A-\lambda I_n)v=0).
  3. Any nonzero vector in that null space is an eigenvector.
• Eigenspace: The eigenspace for (\lambda) is (\ker(A-\lambda I_n)); it includes the zero vector, although the zero vector itself is not an eigenvector.
• Diagonalization link: If (A) has (n) independent eigenvectors, then
  
  TEXT

    A = PDP⁻¹

  
  where columns of (P) are eigenvectors and diagonal entries of (D) are their eigenvalues.

IX. Eigenvalue Properties — Structural Shortcuts

A. properties of eigenvalues

Eigenvalue properties connect spectral information to determinants, traces, matrix powers, and related transformations.

• Sum and product: Counting algebraic multiplicity,
  
  TEXT

    Σᵢ λᵢ = trace(A),      Πᵢ λᵢ = det(A)

  
  Here (\lambda_i) are the (n) eigenvalues, (\operatorname{trace}(A)) is the diagonal sum, and (\det(A)) is the determinant.
• Triangular matrices: The eigenvalues of a triangular or diagonal matrix are its diagonal entries.
• Zero eigenvalue: (0) is an eigenvalue exactly when (A) is singular.
• Transpose and similarity: (A), (A^T), and every similar matrix (P^{-1}AP) have the same characteristic polynomial and eigenvalues.
• Powers and inverse: If (Av=\lambda v), then
  
  TEXT

    Aᵏv = λᵏv;        A⁻¹v = λ⁻¹v

  
  Here (k) is a positive integer; the inverse formula requires (\lambda\ne0).
• Independence property: Eigenvectors corresponding to distinct eigenvalues are linearly independent.
• Multiplicity: Algebraic multiplicity is the root multiplicity in the characteristic polynomial; geometric multiplicity is (\dim\ker(A-\lambda I_n)) and cannot exceed it.

X. Cayley–Hamilton Theorem — A Matrix Satisfies Its Polynomial

A. Cayley-Hamilton theorem

The Cayley–Hamilton theorem states that every square matrix satisfies its own characteristic equation.

• Formal statement: If the characteristic polynomial of (A) is
  
  TEXT

    p(λ) = det(λIₙ − A)
         = λⁿ + cₙ₋₁λⁿ⁻¹ + ··· + c₁λ + c₀,

  
  then
  
  TEXT

    p(A) = Aⁿ + cₙ₋₁Aⁿ⁻¹ + ··· + c₁A + c₀Iₙ = 0.

  
  Here (c_i) are scalar coefficients and (0) is the zero matrix.
• Substitution rule: Scalar powers of (\lambda) become matrix powers of (A), while the constant term becomes (c_0I_n).
• Worked example: For
  
  TEXT

    A = [1  1]
        [0  2],

  
  the characteristic polynomial is (p(\lambda)=\lambda^2-3\lambda+2). Therefore,
  
  TEXT

    A² − 3A + 2I₂ = 0.

• Inverse consequence: Because (\det(A)\ne0), multiply the identity by (A^{-1}):
  
  TEXT

    A − 3I₂ + 2A⁻¹ = 0
    A⁻¹ = (3I₂ − A)/2.

• Applications: The theorem reduces high matrix powers to combinations of (I_n,A,\ldots,A^{n-1}) and provides polynomial formulas for inverses when (A) is nonsingular.