Unit 1 - Notes

MTH174 6 min read

Unit 1: Matrix Algebra

1. Elementary Operations on a Matrix

Elementary operations (or transformations) are operations performed on the rows or columns of a matrix to simplify it without changing its fundamental properties (like rank or the solution set of corresponding linear equations).

There are three types of elementary row (and column) operations:

Interchange of two rows (or columns):
- Notation: $R_i \leftrightarrow R_j$ (Interchange $i$ -th and $j$ -th rows).
- Column Notation: $C_i \leftrightarrow C_j$ .
Multiplication of a row (or column) by a non-zero scalar:
- Notation: $R_i \rightarrow kR_i$ where $k \neq 0$ .
Addition of a scalar multiple of one row (or column) to another:
- Notation: $R_i \rightarrow R_i + kR_j$ (Add $k$ times the $j$ -th row to the $i$ -th row).

Note: Two matrices $A$ and $B$ are said to be equivalent ( $A \sim B$ ) if one can be obtained from the other by a finite sequence of elementary operations.

2. Rank of a Matrix

The rank of a matrix $A$ , denoted by $\rho(A)$ , is defined as the maximum number of linearly independent rows or columns in the matrix. Alternatively, it is the order of the largest non-zero minor of the matrix.

Finding Rank using Elementary Operations

To find the rank easily, we use elementary operations to reduce the matrix into standard forms:

A. Row Echelon Form

A matrix is in row echelon form if:

All non-zero rows are above any rows of all zeros.
The leading coefficient (the first non-zero number from the left, also called the pivot) of a non-zero row is always strictly to the right of the leading coefficient of the row above it.
All entries in a column below a leading entry are zeros.

Rank Calculation: The rank is exactly the number of non-zero rows in its Row Echelon Form.

B. Normal Form (Canonical Form)

By applying both elementary row and column operations, any $m \times n$ matrix of rank $r$ can be reduced to one of the following forms, known as the Normal Form:
$\begin{bmatrix} I_r & 0 \\ 0 & 0 \end{bmatrix}, \quad \begin{bmatrix} I_r & 0 \end{bmatrix}, \quad \begin{bmatrix} I_r \\ 0 \end{bmatrix}, \quad [I_r]$
where $I_r$ is the identity matrix of order $r$ .
Rank Calculation: The rank is $r$ , the order of the identity matrix obtained.

3. Inverse of a Matrix using Elementary Operations (Gauss-Jordan Method)

The inverse of a square non-singular matrix $A$ (i.e., $|A| \neq 0$ ) can be found using elementary row operations.

Procedure:

Write the matrix $A$ alongside an identity matrix $I$ of the same order to form an augmented matrix: $[A | I]$ .
Apply a sequence of elementary row operations exclusively to transform the matrix $A$ into the identity matrix $I$ .
Apply the exact same operations simultaneously to the identity matrix $I$ .
Once the left side becomes $I$ , the right side will transform into $A^{-1}$ .
$[A | I] \xrightarrow{\text{Row Operations}} [I | A^{-1}]$

Warning: Never mix row and column operations when finding the inverse using this method. Stick to only row operations.

4. Solution of Linear Simultaneous Equations

A system of $m$ linear equations in $n$ variables can be written in matrix form as:
$AX = B$
Where:

$A$ is the $m \times n$ coefficient matrix.
$X$ is the $n \times 1$ column matrix of variables.
$B$ is the $m \times 1$ column matrix of constants.

Consistency of the System (Rouché-Capelli Theorem)

To analyze the system, we form the augmented matrix $[A|B]$ and find its rank.

Consistent System (Solution Exists): If .
- Unique Solution: If $\rho(A) = \rho(A|B) = n$ (number of variables).
- Infinite Solutions: If $\rho(A) = \rho(A|B) < n$ . The system will have $n - r$ free variables (where $r$ is the rank).
Inconsistent System (No Solution): If $\rho(A) \neq \rho(A|B)$ .

Gauss Elimination Method

This method uses elementary row operations to systematically solve the equations:

Write the augmented matrix $[A|B]$ .
Apply elementary row operations to reduce $A$ to an upper triangular matrix (Row Echelon Form).
Rewrite the reduced matrix as a system of equations.
Use back-substitution to find the values of the variables, starting from the last variable and working upwards.

5. Eigenvalues and Eigenvectors

For a square matrix $A$ of order $n$ , a scalar $\lambda$ is called an eigenvalue of $A$ if there exists a non-zero column vector $X$ such that:
$AX = \lambda X$
Here, $X$ is called the eigenvector corresponding to the eigenvalue $\lambda$ .

Finding Eigenvalues

Rewrite the equation as $(A - \lambda I)X = 0$ .
For a non-trivial solution (since $X \neq 0$ ), the determinant of the coefficient matrix must be zero:
$|A - \lambda I| = 0$
This is called the Characteristic Equation of matrix $A$ .
Expanding this determinant yields a polynomial of degree $n$ in terms of $\lambda$ . The roots of this polynomial ( $\lambda_1, \lambda_2, \dots, \lambda_n$ ) are the eigenvalues (or latent roots/characteristic roots).

Finding Eigenvectors

For each eigenvalue $\lambda_i$ , substitute it back into the equation:
$(A - \lambda_i I)X = 0$
Solve this homogeneous system of linear equations to find the non-zero vector $X_i$ .

Properties of Eigenvalues

The sum of the eigenvalues of a matrix is equal to the trace of the matrix (sum of main diagonal elements).
The product of the eigenvalues is equal to the determinant of the matrix ( $|A|$ ).
If $\lambda$ is an eigenvalue of $A$ , then $\lambda^k$ is an eigenvalue of $A^k$ .
If $A$ is non-singular, and $\lambda$ is an eigenvalue of $A$ , then $1/\lambda$ is an eigenvalue of $A^{-1}$ .
The eigenvalues of a triangular or diagonal matrix are simply its diagonal elements.

6. Cayley-Hamilton Theorem

The Cayley-Hamilton theorem bridges matrix algebra and polynomial equations.

Statement:
Every square matrix satisfies its own characteristic equation.

If the characteristic polynomial of an $n \times n$ matrix $A$ is:
$P(\lambda) = |A - \lambda I| = (-1)^n \lambda^n + c_{n-1}\lambda^{n-1} + \dots + c_1\lambda + c_0 = 0$
Then substituting matrix $A$ for $\lambda$ yields the zero matrix $O$ :
$(-1)^n A^n + c_{n-1}A^{n-1} + \dots + c_1 A + c_0 I = O$
(Note: the constant term $c_0$ is multiplied by the identity matrix $I$ )

Applications of the Cayley-Hamilton Theorem

Finding the Inverse of a Matrix:
If $|A| \neq 0$ , multiply the Cayley-Hamilton equation by $A^{-1}$ :
$(-1)^n A^{n-1} + c_{n-1}A^{n-2} + \dots + c_1 I + c_0 A^{-1} = O$
Rearranging this gives an explicit formula for $A^{-1}$ in terms of lower powers of $A$ :
$A^{-1} = \frac{-1}{c_0} \left( (-1)^n A^{n-1} + c_{n-1}A^{n-2} + \dots + c_1 I \right)$
(where $c_0 = |A|$ )
Finding Higher Powers of a Matrix:
The theorem allows any matrix polynomial of degree $\ge n$ to be reduced to a polynomial of degree at most $n-1$ .
Example: $A^n = -(-1)^n (c_{n-1}A^{n-1} + \dots + c_1 A + c_0 I)$ .
By multiplying by $A$ , one can recursively find $A^{n+1}, A^{n+2}$ , etc., without performing heavy matrix multiplications.

Unit 2