Simplifying the Gaussian Formula Using Completing the Square

Introduction

As math enthusiasts, we often encounter complex formulas that appear daunting at first glance. However, even the most intricate equations can be simplified and understood. Today, we'll delve into the Gaussian formula and demonstrate how completing the square can unravel its complexity, paving the way for a deeper comprehension of its underlying principles.

1. Unveiling the Components

Let's begin by dissecting the Gaussian formula:

$$e^{-\frac{1}{2}x^T A x + b^T x}$$

Two key components stand out within this equation: a quadratic term and a linear term. Here A denotes a symmetric positive definite matrix, while b represents a vector.

Quadratic Term:

$$-\frac{1}{2}x^T A x$$

Linear Term:

$$b^T x$$

2. Mastering Completing the Square

Completing the square serves as our trusty tool in simplifying quadratic expressions. Our mission is to rewrite the quadratic term plus the linear term in a manner reminiscent of a squared norm. The technique involves strategic manipulation, resulting in a form that's easier to digest. Behold the completing the square formula for our context:

$$-\frac{1}{2}x^T A x + b^T x = -\frac{1}{2}(x - A^{-1}b)^T A (x - A^{-1}b) - \frac{1}{2}b^T A^{-1} b$$

3. Unraveling the Derivation

Let's embark on the journey of derivation:

• We commence with the original quadratic and linear terms.

• By introducing and subtracting a carefully chosen term, we set the stage for transformation.

• Leveraging the properties of matrices and vectors, we rearrange the equation to unveil its true potential.

• Through meticulous rearrangement, the quadratic form metamorphoses into a perfect square.

4. Reintegration into the Exponential Realm

Now, it's time to reintegrate our newfound wisdom into the original Gaussian formula:

$$e^{-\frac{1}{2}x^T A x + b^T x} = e^{-\frac{1}{2}(x - A^{-1}b)^T A (x - A^{-1}b) - \frac{1}{2}b^T A^{-1} b}$$

This transformation yields a revelation:

$$e^{-\frac{1}{2}(x - A^{-1}b)^T A (x - A^{-1}b)} \cdot e^{-\frac{1}{2}b^T A^{-1} b}$$

5. Illuminating Insights

In essence, our exploration showcases the Gaussian formula's underlying structure. It emerges as a Gaussian distribution centred at A⁻¹b, adorned with a covariance matrix A⁻¹. Moreover, the following term assumes the role of a normalization constant, adding clarity to its interpretation:

$$e^{\frac{1}{2}b^T A^{-1} b}$$

Conclusion

In conclusion, completing the square empowers us to unravel the intricacies of mathematical expressions, shedding light on their inner workings. Armed with this knowledge, we embark on a journey of mathematical discovery, fueled by curiosity and the relentless pursuit of understanding.

Let's continue our mathematical odyssey, one equation at a time.