Matrix Inversion

Pre-requisite

To develop an intuition for Matrix Inversion it is important to be familiar with the concepts of Linear Transformation and Matrix Determinants. Please read these articles if you need a refresher.

Core

The Inverse of a matrix depends on a very important condition that the linear transformation in question should not transform the space into a lower dimension. As we have learnt in the Matrix Determinants that such matrices have a determinant of zero.

Let's begin our understanding with the simple case where the determinant of the Matrix is non-zero. Consider an arbitrary Matrix A which transforms the purple vectors into the orange vectors

Then an inverse of Matrix A will restore the original position of the vectors, transforming orange vectors back into purple vectors.

$$\text{The inverse of Matrix A is denoted by } A^{-1}$$

Sure, it's easy to imagine them going back to their original position. But how does that work? What is the inverse of A?

Well, the inverse of A is a separate matrix in itself and like any other matrix, is a linear transformation that does its job of transforming the vector space around it.

The unique property of this matrix is that it is responsible for undoing what Matrix A performs. This is represented theoretically in the form of matrix multiplication. And as we know matrix multiplication is not commutative so we need to be careful of the order in which we express the inverse of A.

To represent the transformations in the above image, we need to check the order in which they were applied. In theory, the associativity of matrix multiplication is from right to left, therefore we write the first Matrix to the right and the second Matrix to the left as shown below

$$A^{-1} \times A$$

And since the vectors end up back where they started, the resultant of this multiplication should also have no effect. In the world of Matrices, we know that this is the identity matrix. Therefore...

$$A^{-1} \times A = I$$

Alternate Reasoning (optional)

There is an alternate reasoning that ends up with the same conclusion as above but it begins with something we already know. You can skip to the "How to" section if you want.

Inversion of multiplication is a division

$$\begin{array}{l} 3 \times 2 = 6 \\ 6 \div 2 = 3 \\ \text{which can also be written as} \\ 6 \times \frac{1}{2} = 3 \\ \text{where } \frac{1}{2} \text{ is the inverse of 2} \end{array}$$

Inversion of addition is a subtraction

$$\begin{array}{l} 3 + 2 = 5 \\ 5 - 2 = 3 \\ \text{which can also be written as} \\ 5 + (-2) = 3 \\ \text{where -2 is the inverse of 2} \end{array}$$

• But why is 1/2 the inverse of 2 in the case of multiplication?
  And why is -2 the inverse of 2 in the case of addition?
  What is the reason that makes it obvious?

• Inverse means to be the opposite or contrary in nature to something else. But if they are opposites, upon combining they should nullify each other. In other words, nothing should change or the state should remain IDENTICAL.

• When you multiply 2 by 1/2 it results in 1 which means nothing has changed.
  Hence, the Multiplicative identity is 1.

• When you add 2 with -2 it results in 0 which also means nothing has changed.
  Hence, the Additive identity is 0.

• The same applies to matrices but in the world of matrices, the identity is not scalar, it is an Identity Matrix!

• Therefore, the cross product of a Matrix with its Inverse should result in an identity matrix as shown below.

  $$\begin{array}{} A \times A^{-1} = I \\ \text{where } A^{-1} \text{ is the inverse of matrix A} \end{array}$$

How to find the inverse?

1. Append the Matrix in question with its Identity Matrix to create an Augmented Matrix

2. Perform row operations on it such that the LHS of the Augmented Matrix becomes the Identity Matrix and the resulting RHS will be the Inverse of the original Matrix

3. (Optional) To verify the solution we can multiply the inverse and the original Matrices and that should give us the Identity Matrix

Note: If you are unfamiliar with row operations or need a quick revision, you can checkout this article

Example (2 x 2 Matrix):

Find the inverse of the following matrix

$$\begin{array}{} A = \begin{bmatrix} 2 & 0 \\ 1 & 4 \end{bmatrix} \end{array}$$

Step 1: Augment with Identity Matrix

$$\left[\begin{array}{rr|rr} 2 & 0 & 1 & 0 \\ 1 & 4 & 0 & 1 \\ \end{array}\right]$$

Step 2: Perform Row Operations

$$\left[\begin{array}{rr|rr} 1 & 0 & \frac{1}{2} & 0 \\ 1 & 4 & 0 & 1 \\ \end{array}\right] R_1 = R_1 \div 2$$

$$\left[\begin{array}{rr|rr} 1 & 0 & \frac{1}{2} & 0 \\ 0 & 4 & -\frac{1}{2} & 1 \\ \end{array}\right] R_2 = R_2 - R_1$$

$$\left[\begin{array}{rr|rr} 1 & 0 & \frac{1}{2} & 0 \\ 0 & 1 & -\frac{1}{8} & \frac{1}{4} \\ \end{array}\right] R_2 = R_2 \div 4$$

$$\begin{array}{l} \therefore A^{-1} = \left[\begin{array}{rr|rr} \frac{1}{2} & 0 \\ -\frac{1}{8} & \frac{1}{4} \\ \end{array}\right] \end{array}$$

Step 3: Verify (Optional)

Multiplying Matrix A with its inverse should give us an identity matrix

$$\begin{array}{l} A \times A^{-1} = \begin{bmatrix} 2 & 0 \\ 1 & 4 \end{bmatrix} \times \begin{bmatrix} \frac{1}{2} & 0 \\ -\frac{1}{8} & \frac{1}{4} \end{bmatrix} \\ \quad\quad\quad\quad = \begin{bmatrix} 2 \times \frac{1}{2} + 0 & 0 + 0 \\ 1 \times \frac{1}{2} + 4 \times -\frac{1}{8} & 1 \times 0 + 4 \times \frac{1}{4} \end{bmatrix} \\ \quad\quad\quad\quad = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \end{array}$$

Hence, our solution is verified!

Example (3 x 3 Matrix):

The steps remain exactly the same for a 3x3 Matrix with no changes. You can try solving this question yourself and check your solution here.

Find the inverse of...

$$B = \left[\begin{array}{r} 3 & 0 & 2 \\ 2 & 0 & -2 \\ 0 & 1 & 1 \\ \end{array}\right]$$

Augment with Identity Matrix

$$\left[\begin{array}{rrr|rrr} 3 & 0 & 2 & 1 & 0 & 0 \\ 2 & 0 & -2 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 & 0 & 1 \\ \end{array}\right]$$

Perform Row Operations

$$\begin{array}{l} = \left[\begin{array}{rrr|rrr} 3 & 0 & 2 & 1 & 0 & 0 \\ 2 & 0 & -2 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 & 0 & 1 \\ \end{array}\right] \\ \\ = \left[\begin{array}{rrr|rrr} 5 & 0 & 0 & 1 & 1 & 0 \\ 2 & 0 & -2 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 & 0 & 1 \\ \end{array}\right] R_1 = R_1 + R_2 \\ \\ = \left[\begin{array}{rrr|rrr} 1 & 0 & 0 & 0.2 & 0.2 & 0 \\ 0 & 0 & -2 & -0.4 & 0.6 & 0 \\ 0 & 1 & 1 & 0 & 0 & 1 \\ \end{array}\right] \begin{array}{l} R_1 = R_1 \div 5\\ R_2 = R_2 - 2R_1 \end{array} \\ \\ = \left[\begin{array}{rrr|rrr} 1 & 0 & 0 & 0.2 & 0.2 & 0 \\ 0 & 0 & 1 & 0.2 & 0.3 & 0 \\ 0 & 1 & 0 & -0.2 & 0.3 & 1 \\ \end{array}\right] \begin{array}{l} R_2 = R_2 \div -2\\ R_3 = R_3 - R_2 \end{array} \\ \\ = \left[\begin{array}{rrr|rrr} 1 & 0 & 0 & 0.2 & 0.2 & 0 \\ 0 & 1 & 0 & -0.2 & 0.3 & 1 \\ 0 & 0 & 1 & 0.2 & 0.3 & 0 \\ \end{array}\right] R_2 \iff R_3 \end{array}$$

Therefore, the inverse of Matrix B is

$$B^{-1} = \left[ \begin{array}{r} 0.2 & 0.2 & 0 \\ -0.2 & 0.3 & 1 \\ 0.2 & 0.3 & 0 \\ \end{array} \right]$$

I'll encourage you to multiply the inverse with B and verify the solution as well

---

Thanks for reading! You can read the next article in the series which explains the process of Gaussian Reduction

If this article has helped you in any form you can show your appreciation by reacting to this article and if you have any feedback please feel free to leave them in the comments below.

Thanks again!