Understanding Error Metrics in Regression: MSE, MAE, and RMSE

When building predictive models, especially in regression tasks, evaluating their performance is crucial. This is where error metrics come into play. In this blog post, we will explore three common error metrics: Mean Squared Error (MSE), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE). We will discuss their definitions, formulas, examples, and when to use each one.

1. Mean Squared Error (MSE)

Definition

Mean Squared Error (MSE) is a popular cost function used in regression analysis. It measures the average of the squares of the errors—that is, the average squared differences between the predicted and actual values. By squaring the errors, MSE becomes sensitive to larger discrepancies (outliers), heavily penalizing larger deviations.

Formula

The formula for calculating MSE is:

MSE=1n∑i=1n(yi−y^i)2$.

Where:

$$y_i = \text{actual value for the } i\text{-th data point}$$

$$\hat{y}_i = \text{predicted value for the } i\text{-th data point} \\$$

$$n = \text{total number of data points}$$

Example

Let’s consider a scenario where we want to predict house prices. We have the actual prices and predicted prices for three houses:

Step 1: Calculate the Squared Errors

• For House 1: (200,000−210,000)² = 100,000,000

• For House 2: (300,000−295,000)² = 25,000,000

• For House 3: (250,000−240,000)² = 100,000,000

Step 2: Find the Average

$$\text{MSE} = \frac{100,000,000 + 25,000,000 + 100,000,000}{3} = 75,000,000$$

Interpretation

A lower MSE indicates that the model's predictions are closer to the actual values. However, because of the squaring of errors, it tends to penalize larger errors more significantly, which might not always be desirable.

2. Mean Absolute Error (MAE)

Definition

Mean Absolute Error (MAE) is another cost function that measures the average of the absolute differences between predicted and actual values. Unlike MSE, MAE does not square the errors, making it less sensitive to outliers and easier to interpret.

Formula

The formula for calculating MAE is:

$$\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|$$

Example

Using the same house price example as above, let’s calculate the absolute errors:

Step 1: Calculate the Absolute Errors

• For House 1: | 200,000 - 210,000 | = 10,000

• For House 2: | 300,000 - 295,000 | = 5,000

• For House 3: | 250,000 - 240,000 | = 10,000

Step 2: Find the Average

$$\text{MAE} = \frac{10,000 + 5,000 + 10,000}{3} \approx 8,333.33$$

Interpretation

MAE provides a direct measure of the average error in the same units as the target variable (e.g., dollars). It is easier to interpret and less sensitive to outliers, making it a suitable choice when outliers are not significant.

3. Root Mean Squared Error (RMSE)

Definition

Root Mean Squared Error (RMSE) is similar to MSE, but it takes the square root of the average squared errors. This scaling back allows RMSE to be in the same unit as the target variable, making it more interpretable while still being sensitive to outliers.

Formula

The formula for calculating RMSE is:

$$\text{RMSE} = \sqrt{\text{MSE}} = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2}$$

Example

Using the MSE we calculated earlier:

RMSE=75,000,000≈8,660.25\text{RMSE} = \sqrt{75,000,000} \approx 8,660.25RMSE=75,000,000​≈8,660.25

$$\text{RMSE} = \sqrt{75,000,000} \approx 8,660.25$$

Interpretation

RMSE provides an error measure in the same unit as the target variable (like dollars for house prices). It is more intuitive than MSE but still penalizes larger errors due to the squaring of deviations.

Summary Comparison

When to Use Each Metric

• Use MSE when you want to heavily penalize larger errors.

• Use MAE if you need a balanced approach that’s less sensitive to outliers.

• Use RMSE for a more interpretable version of MSE, where the units match the target variable.

Residuals and Cost Functions? 🤔

Have you ever wondered what residuals and cost functions are and how they differ? These concepts are essential in regression analysis and can significantly impact your understanding of model performance.

To learn more about these important terms and their differences, check out my blog on the Difference between Residual and Cost Function.

Conclusion

Each cost function serves different purposes depending on the nature of the data and the goals of the model. Understanding these error metrics is essential for evaluating and improving the performance of your regression models. Choose the one that best aligns with your modeling goals and the characteristics of your dataset.