Machine Learning Model Fit

In our previous post we briefed about the Machine Learning development lifecycle and touched upon ML models performance evaluation. In continuation to that, we will see how and why model fit is important aspect of model evaluation.

If you recall, we presented our ML model with Training set, Validation set and Test set. Training set is the dataset used to train the model. Validation and tests sets are the ones that we will use to evaluate the trained model performance.

Consider we have dataset with attribute X and Y. X is the value/input example and Y is the response/target corresponding to the X value. During training, when training set it fed to the model, it tries to create a regression curve to fit the data points on XY graph.

Imagine a curve passing through every data point on the X-Y plot as shown below.

Overfitting

Overfitting is when the model performs well on the training data but does not perform well on the evaluation data. This is because the model memorized the data it has seen and is unable to generalize to unseen examples.

This is a higher degree polynomial trying to fit almost each data point on the plot. In easy language, model is trying to remember each instance (pair of X-Y) of data pair provided in training dataset. Meaning the error during the training will be minimum or ~0.

Underfitting

Underfitting is when the model performs poorly on the training data. This is because the model is unable to capture the relationship between the input examples (often called X) and the target values (often called Y).

If your model is underfitting and performing poorly on the training data, it could be that the model is too simple (the input features are not expressive enough) to describe the target well.

Here you will see that the straight line is not able to capture the relationship of the dataset (X-Y pair). Striaght line P(x)= ax + b is polynomial of degree one.

Now that you have seen how graphically overfitting and underfitting looks like and thier disadvantages. What is the solution to this problem? Should we have higher degree polynomial or lower degree polynomial in order to capture the best of both worlds?

It is Balanced model.

Balanced

The model is balanced when it is not overfit or underfit to the training data.

A "balanced model" in the context of polynomials refers to a polynomial function where the coefficients of different powers of the variable are chosen in a way that prevents the model from being overly influenced by any single term, achieving a good balance between fitting the data accurately and avoiding overfitting; essentially, it means not relying too heavily on high-degree terms while still capturing the necessary curvature in the data.

Key points about Balanced model:

Addressing class imbalance:

The primary goal of a balanced model is to mitigate the issue of biased predictions that can occur when a model is trained on imbalanced data, where it might primarily focus on the majority class.

Resampling techniques:

To achieve balance, common methods include:

• Oversampling: Duplicating data points from the minority class to increase its representation.

• Undersampling: Removing data points from the majority class to reduce its representation.

Class weights:

Another approach is to assign higher weights to the minority class during training, giving the model a stronger incentive to learn from its examples.

Evaluation metrics:

When assessing a balanced model, metrics like F1-score are preferred over simple accuracy as they consider both precision and recall, providing a more comprehensive view of performance across all classes.

In addition to model fit, Bias and Variance are also important aspect that contribute to errors the model makes on unseen data. I will cover bias and variance in my next article and will establish the relationship among bias, variance and model fitting.

---

Thanks for reading this article. For any suggestions please leave comment.