Introduction

Binary classification is one of the most fundamental tasks in machine learning. Whether you're predicting if an email is spam or not, detecting fraudulent transactions, or diagnosing whether a tumor is malignant or benign — all of these are binary classification problems. The goal is simple: classify input data into one of two categories.

In this blog, we'll walk through how to build, train, and evaluate a binary classification model using PyTorch, one of the most popular deep learning frameworks. You'll learn how to:

• Prepare and load your data

• Build a custom neural network

• Train it using backpropagation and gradient descent

• Evaluate its accuracy on unseen data

• Visualize the decision boundary

• 

By the end of this tutorial, you'll have a complete working pipeline — from data generation to model evaluation — and a solid understanding of how binary classification works under the hood.

Whether you're a beginner taking your first steps into deep learning or someone looking to brush up on PyTorch basics, this guide is crafted to be clear, concise, and hands-on.

Let’s dive in!

Creating a Synthetic Binary Classification Dataset

Before diving into neural networks, we need a dataset to work with. For this tutorial, we'll use scikit-learn’s make_classification function to generate a synthetic dataset. This is a handy utility that allows you to create a labeled dataset with just a few lines of code.

We'll generate 1,000 samples, each having 2 input features, and assign them to one of two classes (0 or 1). These features will represent points in 2D space — perfect for visualizing decision boundaries later on.

Here’s how you can generate the data:

from sklearn.datasets import make_circles

n_samples = 10000

X, y = make_circles(n_samples,
                    noise=0.1,
                    random_state=41)

This will give us:

• X: a NumPy array of shape (1000, 2) containing the feature values

• y: a NumPy array of shape (1000,) containing the binary labels (0 or 1)

Visualizing the Synthetic Data

Before training a model, it's a good idea to visualize the data to get an intuitive understanding of what we're working with. Since our dataset has two features (x1 and x2), we can easily plot them in a 2D space and color the points based on their class labels.

We'll use matplotlib to create a scatter plot where:

• The x-axis represents x1

• The y-axis represents x2

• Colors indicate the binary class (0 or 1)

Assuming you’ve stored your data into a pandas DataFrame for convenience, here's how to create the plot:

import matplotlib.pyplot as plt
import pandas as pd

# Create a DataFrame for visualization
data = pd.DataFrame(X, columns=['x1', 'x2'])
data['label'] = y

# Scatter plot of the two classes
plt.figure(figsize=(8, 6))
plt.scatter(x=data['x1'],
            y=data['x2'],
            c=data['label'],
            cmap=plt.cm.RdYlBu,
            edgecolor='k',
            alpha=0.7)

plt.title('Synthetic Binary Classification Dataset')
plt.xlabel('Feature x1')
plt.ylabel('Feature x2')
plt.grid(True)
plt.show()

This plot provides a clear visual separation of the two classes and helps us anticipate how a model might draw a decision boundary between them.

Preparing the Data for PyTorch

Now that we’ve generated and visualized our synthetic dataset, it’s time to prepare it for use with PyTorch. This involves converting the data from NumPy arrays to PyTorch tensors, and then splitting it into training and test sets.

Here's how you can do it:

import torch
from sklearn.model_selection import train_test_split

# Convert NumPy arrays to PyTorch tensors
X = torch.from_numpy(X).type(torch.float)
y = torch.from_numpy(y).type(torch.float)

# Split the dataset into training and testing sets (80% train, 20% test)
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=41)

Why .type(torch.float)?

By default, NumPy creates arrays in float64 (double precision), but PyTorch uses float32 (single precision) for most models and operations. This mismatch can lead to subtle bugs or performance issues during training, especially on GPU.

To ensure compatibility with PyTorch layers (like nn.Linear) and loss functions, we explicitly convert the data using:

.type(torch.float)

Building the Neural Network Model

With our data prepared, it's time to build a neural network that can learn to classify inputs into one of two binary classes. We'll use PyTorch’s nn.Module class to define a custom feedforward neural network.

Here’s the full code for our model:

import torch.nn as nn

class DemoModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.layer_1 = nn.Linear(in_features=2, out_features=10)
        self.layer_2 = nn.Linear(in_features=10, out_features=10)
        self.layer_3 = nn.Linear(in_features=10, out_features=1)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.relu(self.layer_1(x))
        x = self.relu(self.layer_2(x))
        x = self.layer_3(x)
        return x

# Move model to GPU if available
model = DemoModel().to(device=device)

Understanding the Architecture:

• Input layer: Takes in 2 features (since our data points are 2D).

• Hidden layer 1: A linear transformation with 10 neurons followed by a ReLU activation.

• Hidden layer 2: Another linear layer with 10 neurons and ReLU.

• Output layer: A final linear layer that outputs a single value (logit), which we’ll later use with a sigmoid function to produce a probability.

Why ReLU?

ReLU (Rectified Linear Unit) introduces non-linearity, helping the model learn complex patterns in the data. Without it, your model would behave like a simple linear function regardless of how many layers you stack.

This is a typical structure for a binary classifier — input → hidden layers → output — with ReLU activations in between.

Inspecting Model Parameters with state_dict()

Once the model is defined, it's often useful to inspect or save its parameters. In PyTorch, every nn.Module (including our DemoModel) has a built-in method called state_dict().

pythonCopyEditmodel.state_dict()

This returns an ordered dictionary containing all the learnable parameters of the model — such as the weights and biases of each layer. For example, it will show entries like:

• layer_1.weight

• layer_1.bias

• layer_2.weight

• and so on...

These parameters are what the optimizer updates during training.

Why is this useful?

• You can save the model weights using torch.save(model.state_dict(), 'model.pth')

• You can load them later into the same model architecture with model.load_state_dict(...)

• It helps in debugging or visualizing what the model has learned so far.

Defining the Loss Function and Optimizer

Now that our model is built, the next step is to define how it learns. This involves setting up:

• A loss function: to measure how far the model's predictions are from the true labels

• An optimizer: to update model weights in the right direction based on the computed loss

Here’s the code we use:

import torch.nn as nn
import torch.optim as optim

# Binary Cross Entropy loss with built-in sigmoid
loss_fn = nn.BCEWithLogitsLoss()

# Stochastic Gradient Descent optimizer
optimizer = optim.SGD(params=model.parameters(), lr=0.01)

Why BCEWithLogitsLoss() Instead of BCELoss()?

PyTorch provides two similar loss functions for binary classification:

BCEWithLogitsLoss() = Sigmoid + BCELoss combined into one function.

# Equivalent to:
nn.Sequential(
    nn.Sigmoid(),
    nn.BCELoss()
)

Benefits of BCEWithLogitsLoss():

• Numerical stability: Avoids issues with extreme input values (logits)

• Better performance: Fewer operations, cleaner backpropagation

• Simpler code: No need to manually apply sigmoid before computing loss

So whenever you're working with raw outputs from the model (called logits, not probabilities), you should use BCEWithLogitsLoss() — which is exactly what our model outputs in the final layer.

Measuring Model Accuracy

In binary classification, accuracy is one of the most intuitive evaluation metrics — it simply tells us what percentage of the predictions were correct.

Since PyTorch doesn’t provide a built-in accuracy function for binary tasks, we’ll define our own:

def accuracy_fn(y_pred, y_true):
    correct = torch.eq(y_true, y_pred).sum().item()  # Count where prediction == true label
    acc = (correct / len(y_pred)) * 100              # Convert to percentage
    return acc

• torch.eq(y_true, y_pred): Compares predicted and true labels element-wise and returns a tensor of boolean values.

• .sum().item(): Counts how many predictions were correct.

• correct / len(y_pred) * 100: Computes the accuracy as a percentage.

• For example:

  • If out of 100 samples, 92 were classified correctly, this function will return 92.0.

This simple helper function is used during both training and evaluation phases to monitor how well our model is learning to separate the two classes.

Training the Model

With our model, loss function, and optimizer in place, we’re ready to train the neural network! This is where the magic happens — the model learns patterns from the training data by minimizing the loss over multiple iterations (epochs).

Here's the full training loop:

# Optional: make CUDA errors more traceable
# os.environ["CUDA_LAUNCH_BLOCKING"] = "1"

# Set random seeds for reproducibility
torch.manual_seed(42)
torch.cuda.manual_seed(42)

epochs = 1000

# Move training and test data to the same device as the model
x_train, x_test = x_train.to(device), x_test.to(device)
y_train, y_test = y_train.to(device), y_test.to(device)

for epoch in range(epochs):
    model.train()

    # Forward pass (train)
    y_logits = model(x_train).squeeze()
    y_pred = torch.round(torch.sigmoid(y_logits))  # logits → probabilities → binary labels

    # Compute training loss and accuracy
    loss = loss_fn(y_logits, y_train)
    acc = accuracy_fn(y_true=y_train, y_pred=y_pred)

    # Backpropagation
    optimizer.zero_grad()  # reset gradients
    loss.backward()        # compute gradients
    optimizer.step()       # update model weights

    # Evaluation on test set (inference mode for speed)
    model.eval()
    with torch.inference_mode():
        test_logits = model(x_test).squeeze()
        test_pred = torch.round(torch.sigmoid(test_logits))

        test_loss = loss_fn(test_logits, y_test)
        test_acc = accuracy_fn(y_true=y_test, y_pred=test_pred)

    # Logging every 5 epochs
    if epoch % 5 == 0:
        print(f"Epoch: {epoch} | Train Loss: {loss:.5f}, Train Acc: {acc}% | Test Loss: {test_loss:.5f} | Test Acc: {test_acc}%")

What’s Happening in the Loop?

• Training Mode: model.train() turns on dropout, batchnorm, etc. (if used).

• Forward Pass: The model outputs logits (raw scores before sigmoid).

• Prediction: We apply torch.sigmoid() to convert logits to probabilities and then round them to 0 or 1.

• Loss & Accuracy: We calculate how well the model performed using Binary Cross Entropy Loss and a custom accuracy function.

• Backpropagation: Standard PyTorch pattern — zero out gradients, calculate gradients, step the optimizer.

• Evaluation Mode: model.eval() disables layers like dropout; torch.inference_mode() disables autograd for faster computation and lower memory usage.

• Logging: Every 5 epochs, we print the training and testing loss and accuracy to track progress.

Conclusion

In this tutorial, we walked step-by-step through the process of building a binary classification model using PyTorch. Starting from data generation and preprocessing, we built a simple neural network, trained it with gradient descent, and evaluated its performance.

We also explored important concepts like:

• Why BCEWithLogitsLoss() is preferred over BCELoss()

• How to use .state_dict() to inspect model parameters

• Writing custom metrics like accuracy

By now, you should have a solid foundation for working with binary classification problems in PyTorch.