I. Orientation

Matrix algebra studies rectangular arrays of numbers and the operations used to represent linear transformations and systems of simultaneous equations. Its central principle is that algebraic operations on rows and columns can reveal structural information without changing the essential solution set of a system. A matrix is commonly written as (A=[a{ij}]{m\times n}), where (m) is the number of rows, (n) is the number of columns, and (a_{ij}) is the entry in row (i), column (j).

• Matrix equality: Two matrices are equal only when they have the same order and corresponding entries are equal.
• Matrix product: If (A) is (m\times n) and (B) is (n\times p), then (AB) exists and is (m\times p); generally, (AB\ne BA).
• Identity matrix: (I_n) is the (n\times n) matrix with ones on the principal diagonal and zeros elsewhere; (AI_n=I_nA=A).
• Determinant condition: A square matrix (A) is nonsingular precisely when (\det(A)\ne0).
• Elementary row operations: Interchanging rows, multiplying a row by a nonzero scalar, and adding a multiple of one row to another preserve row-equivalence.
• Notation convention: (A^T) denotes transpose, (\det(A)) the determinant, (I) the identity matrix, and (0) a zero matrix or scalar as appropriate.

II. Elementary Operations and Rank

A. Elementary operations and their use in getting the rank of a matrix

Elementary operations convert a matrix into a simpler row-echelon form, while the rank measures the number of independent rows or columns.

• Row interchange: (R_i\leftrightarrow R_j) exchanges two rows; for example, it can move a nonzero entry into a pivot position.
• Row scaling: (R_i\to kR_i), where (k\ne0), multiplies a row by a nonzero constant and does not change rank.
• Row replacement: (R_i\to R_i+kR_j) adds (k) times row (j) to row (i); this eliminates entries below or above pivots.
• Rank definition: (\operatorname{rank}(A)) is the maximum number of linearly independent rows or columns of (A).
• Echelon criterion: After reduction to row-echelon form, rank equals the number of nonzero rows, equivalently the number of pivots.
• Reduced echelon form: In reduced row-echelon form, each pivot is (1) and is the only nonzero entry in its column; this is useful for solving systems and finding inverses.
• Worked example: For
  [
  A=\begin{bmatrix}1&2&3\2&4&6\1&1&1\end{bmatrix},
  ]
  apply (R_2\to R_2-2R_1) and (R_3\to R_3-R_1):
  [
  \begin{bmatrix}1&2&3\0&0&0\0&-1&-2\end{bmatrix}.
  ]
  There are two nonzero rows, so (\operatorname{rank}(A)=2).
• Practical limitation: Numerical round-off can make a theoretically zero pivot appear as a small nonzero number; pivoting and a suitable tolerance are used in computational work.

III. Inverse of a Matrix

A. Inverse of a matrix

The inverse reverses multiplication by a nonsingular square matrix: for a square matrix (A), its inverse (A^{-1}) satisfies (AA^{-1}=A^{-1}A=I).

• Existence condition: (A^{-1}) exists if and only if (\det(A)\ne0), or equivalently (\operatorname{rank}(A)=n) for an (n\times n) matrix.
• Two-by-two formula: If
  [
  A=\begin{bmatrix}a&b\c&d\end{bmatrix},
  ]
  then
  [
  A^{-1}=\frac{1}{ad-bc}\begin{bmatrix}d&-b\-c&a\end{bmatrix},
  ]
  where (a,b,c,d) are entries of (A).
• Adjoint formula: For any nonsingular square matrix,
  [
  A^{-1}=\frac{\operatorname{adj}(A)}{\det(A)},
  ]
  where (\operatorname{adj}(A)) is the transpose of the cofactor matrix.
• Gauss–Jordan method: Form the augmented matrix ([A\mid I]), reduce it by elementary row operations, and obtain ([I\mid A^{-1}]).
• Worked example:
  [
  A=\begin{bmatrix}2&1\1&1\end{bmatrix},\qquad \det(A)=2(1)-1(1)=1.
  ]
  Therefore,
  [
  A^{-1}=\begin{bmatrix}1&-1\-1&2\end{bmatrix}.
  ]
• Properties: ((AB)^{-1}=B^{-1}A^{-1}), ((A^T)^{-1}=(A^{-1})^T), and ((A^{-1})^{-1}=A), whenever the stated inverses exist.
• Interpretation: The inverse exists only when the transformation represented by (A) is one-to-one and does not collapse independent directions into dependence.

IV. Solution of Linear Simultaneous Equations

A. Solution of linear simultaneous equations

A linear system expresses several linear equations in unknowns (x_1,\ldots,x_n) and can be written compactly as (AX=B), where (A) is the coefficient matrix, (X) the column of unknowns, and (B) the constant column.

• Matrix representation:
  [
  \begin{bmatrix}
  a{11}&\cdots&a{1n}\
  \vdots&\ddots&\vdots\
  a{m1}&\cdots&a{mn}
  \end{bmatrix}
  \begin{bmatrix}x_1\\vdots\x_n\end{bmatrix}

  \begin{bmatrix}b_1\\vdots\b_m\end{bmatrix}.
  ]

• Augmented matrix: The system (AX=B) is represented by ([A\mid B]); row operations simplify the equations without changing their solution set.
• Unique solution: For (n) unknowns, a unique solution occurs when (\operatorname{rank}(A)=\operatorname{rank}([A\mid B])=n).
• Infinitely many solutions: This occurs when (\operatorname{rank}(A)=\operatorname{rank}([A\mid B])<n); at least one variable is free.
• No solution: This occurs when (\operatorname{rank}(A)<\operatorname{rank}([A\mid B])), producing a contradiction such as (0=5).
• Cramer’s rule: For a square system with (\det(A)\ne0),
  [
  x_i=\frac{\det(A_i)}{\det(A)},
  ]
  where (A_i) is formed by replacing column (i) of (A) by (B).
• Inverse method: If (A) is nonsingular, then (AX=B) gives
  [
  X=A^{-1}B.
  ]
• Worked example: For (x+y=5) and (x-y=1),
  [
  A=\begin{bmatrix}1&1\1&-1\end{bmatrix},\quad
  B=\begin{bmatrix}5\1\end{bmatrix}.
  ]
  Adding the equations gives (2x=6), hence (x=3); substitution gives (y=2).
• Homogeneous systems: In (AX=0), the zero solution always exists; nonzero solutions occur when (\operatorname{rank}(A)<n), equivalently when a square (A) is singular.

V. Eigen-values and Eigenvectors

A. Eigen-values and eigenvectors of a matrix

For a square matrix (A), a nonzero vector (X) is an eigenvector corresponding to eigen-value (\lambda) when (AX=\lambda X). Eigenvectors preserve direction under the transformation, while (\lambda) gives the associated scale factor.

• Characteristic equation: Rearranging (AX=\lambda X) gives
  [
  (A-\lambda I)X=0.
  ]
  A nonzero (X) exists only when
  [
  \det(A-\lambda I)=0.
  ]
• Characteristic polynomial: (p(\lambda)=\det(\lambda I-A)) is a polynomial of degree (n) for an (n\times n) matrix.
• Finding eigenvectors: After finding an eigen-value (\lambda), solve ((A-\lambda I)X=0). Every nonzero scalar multiple of an eigenvector is also an eigenvector.
• Algebraic multiplicity: The number of times (\lambda) occurs as a root of the characteristic polynomial.
• Geometric multiplicity: The dimension of the null space of (A-\lambda I); it cannot exceed algebraic multiplicity.
• Worked example: Let
  [
  A=\begin{bmatrix}2&1\1&2\end{bmatrix}.
  ]
  Then
  [
  \det(A-\lambda I)=(2-\lambda)^2-1
  =(\lambda-3)(\lambda-1).
  ]
  Eigen-values are (3) and (1). For (\lambda=3), solving ((A-3I)X=0) gives (X=(1,1)^T). For (\lambda=1), an eigenvector is (X=(1,-1)^T).
• Diagonalization: If (A) has (n) linearly independent eigenvectors, then (A=PDP^{-1}), where (P) contains eigenvectors as columns and (D) is diagonal with corresponding eigen-values.
• Important properties:
  • The sum of eigen-values equals (\operatorname{tr}(A)), the sum of diagonal entries.
  • Their product equals (\det(A)).
  • Eigen-values of a triangular matrix are its diagonal entries.
  • Eigen-values of (A^{-1}), when it exists, are (1/\lambda).

VI. Cayley-Hamilton Theorem

A. Cayley-Hamilton theorem

The Cayley-Hamilton theorem states that every square matrix satisfies its own characteristic equation. If (p(\lambda)=\det(\lambda I-A)), then replacing (\lambda) by (A) gives (p(A)=0), where (0) is the zero matrix.

• Formal statement: If
  [
  p(\lambda)=\lambda^n+c1\lambda^{n-1}+\cdots+c{n-1}\lambda+c_n,
  ]
  is the characteristic polynomial of (A), then
  [
  p(A)=A^n+c1A^{n-1}+\cdots+c{n-1}A+c_nI=0.
  ]
• Meaning of substitution: The scalar powers (\lambda^k) become matrix powers (A^k), the scalar constant becomes a multiple of (I), and the result is the zero matrix.
• Worked example: For
  [
  A=\begin{bmatrix}2&1\1&2\end{bmatrix},
  ]
  the characteristic polynomial is
  [
  p(\lambda)=\lambda^2-4\lambda+3.
  ]
  Hence Cayley-Hamilton gives
  [
  A^2-4A+3I=0.
  ]
  Therefore (A^2=4A-3I), allowing all higher powers of (A) to be reduced to a combination of (A) and (I).
• Finding the inverse: If the constant term (c_n\ne0), rearrange:
  [
  A^n+c1A^{n-1}+\cdots+c{n-1}A=-c_nI.
  ]
  Multiplication by (A^{-1}) yields an expression for (A^{-1}) in lower powers of (A).
• Computing powers: Relations such as (A^2=4A-3I) replace lengthy multiplication and are useful for (A^5), matrix sequences, and recurrence relations.
• Conceptual significance: The theorem shows that powers (I,A,A^2,\ldots) are linearly dependent after at most (n) steps for an (n\times n) matrix, linking determinants, eigen-values, inverses, and polynomial expressions in matrices.