Introduction

Classification in machine learning uses supervised learning techniques to classify different categories. It uses a training and validation dataset to train data to classify categories into different labels and finally uses test dataset to evaluate the performance of the model.

Building a model that can learn from a set of labeled instances (training data) and then generalize that understanding to precisely predict the labels of new, untainted data (testing data) is the aim of classification.

Now the question arises how can we evaluate the performance of the model?

Evaluation metrics

We evaluate our model using precision , accuracy and f1score for classification problems.

We would take the example of the following code snippet for it

# Import the python libraries
import numpy as np
import pandas as pd

#importing traintestsplit to split the data dor training and testing
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# Load the dataset file
data = pd.read_csv('your_dataset.csv')
# Split the data into features (X) and target (y)
X = data.drop('target_column_name', axis=1)
y = data['target_column_name']

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Here we will use SVM algorithmn for classfication 
# Create an instance of the Support Vector Machine (SVM) classifier
# We can customize the kernel and hyperparameters 
classifier = SVC(kernel='linear', C=1.0, random_state=42)

# Train the model on the training data
classifier.fit(X_train, y_train)
y_pred = classifier.predict(X_test)

# Calculate and explain the classification metrics
accuracy = accuracy_score(y_test, y_pred)
precision = precision_score(y_test, y_pred, average='weighted')
recall = recall_score(y_test, y_pred, average='weighted')
f1 = f1_score(y_test, y_pred, average='weighted')
conf_matrix = confusion_matrix(y_test, y_pred)

# Print the classification report and confusion matrix for more detailed evaluation
print("Classification Report:")
print(classification_report(y_test, y_pred))

print("Confusion Matrix:")
print(confusion_matrix(y_test, y_pred))

Let's discuss the first evaluation parameter, accuracy

Accuracy :

It means total number of correct predictions by our model divided by total predictions or total instances. It takes the number of true positives and true negatives divided by total number of instances .But it is not a good metric for evaluation, Let's say if we are predicting messages to be spam or legitimate, and let's say for the training set , we have 90% of the messages as legitimate and the remaining one as spam.

After training the data we finally predict and compare the X_test to the true y_test, and then in this case we might get high accuracy because it is unbalanced class.

For unbalanced problems, we use other classification metrics.

Let's discuss that

Precision :

Precision could be defined as the number of true positives divided by the number of predicted positives.

Precision = True Positives / ( True Positive + False Positive )

A high precision score means the model is cautious in categorising cases as positive and has a low rate of false positives. Precision becomes a critical parameter to optimise in applications where false positives are expensive such as movie recommendation , email - spam detection where customer retention is essential .

                Actual Positive   Actual Negative
Predicted Positive       45                10
Predicted Negative       10               100

Precision = TP / (TP + FP) = 45 / (45 + 10) = 45 / 55 = 0.81

It means that model predicts 81% of the cases correct positive cases out of total positive cases predicted

Recall :

Recall for a label is defined as the number of true positives divided by the total number of actual positives.

Recall = True Positives / ( True Positive + False Negative )

It is used when we cant bear the cost of false negative , when false negative become expensive as in case of medical disease prediction.

For example , if we take cancer disease prediction model , we cant bear high false negative which means our prediction says that the person doesn't have the disease but actually does have disease . We need to avoid such instances.

               Actual Positive   Actual Negative
Predicted Positive       45                10
Predicted Negative       10               100
Recall = TP / (TP + FN) = 45 / (45 + 10) = 45 / 55 ≈ 0.82

It means our model is identifying 82% of the disease as disease actually present .

F1 Score :

F1 Score is the harmonic mean of precision and recall , it provides a sort of balanced measure between the two

F1 Score = 2 * (Precision * Recall) / (Precision + Recall)

From previous cases

F1 Score = 2 * (Precision * Recall) / (Precision + Recall) 
         = 2 * (0.81 * 0.82) / (0.81 + 0.82)
         ≈ 0.814

It is very widely used when you want to have a balance between precision and recall. Like if we have high precision and low recall or vice versa , when we want to have a balance between two, we can look for F1 score.

Conclusion

To summarise, comprehension of classification metrics is critical for evaluating and optimising machine learning models. We can ensure accurate categorization results by selecting the appropriate metrics and improving model performance. We have investigated the significance of classification metrics and learned how to maximise the potential of our machine learning projects thanks to this detailed guide.

Thanks for reading :) ✨

#machinelearning #machinelearningmodels #python