Table of contents”

Section 1.1 : What is perceptron model

Section 1.2: loss function

Section 1.3: Multi-level Perceptron

Section 1.4: Building model using Perceptron

Perceptron: The Building Blocks of Neural Networks

Hey everyone! Today, let's dive into the fascinating world of neural networks by exploring their fundamental unit: the perceptron.

What Exactly is a Perceptron? 🤔

Perceptron is simply like adding a margin between two different things. It is like a straight line, We find the straight line by training a set of data and when a new input comes , we give our prediction by analyzing whether the input is in the positive side of the straight line or in the negative side. By analyzing this , we give our prediction.

Think of a perceptron as a simple decision-making unit. It takes several inputs, each with an associated weight, and calculates a weighted sum. Then, it adds a bias (like an offset). Finally, it applies an activation function to this sum to produce an output. This output is often a binary value (0 or 1), representing a decision.

Mathematically, it looks something like this:

Output = activation_function( (x1*w1 + x2*w2 + ... + xn*wn) + b )

Where:

x represents the inputs
w represents the weights
b is the bias
activation_function is a function that decides the output

Example: Perceptron in Action

Let’s take a simple example of classifying whether a given fruit is an apple or not based on two inputs: its weight (in grams) and its color (on a scale of 0 to 1, where 1 means red). The perceptron receives these inputs, multiplies them by their weights, adds a bias, and applies the activation function to decide whether the fruit is an apple or not.

Input 1 (Weight): 150 grams
Input 2 (Color): 0.9 (since the fruit is mostly red)
Weights: [0.5, 1.0]
Bias: 1.5

The perceptron’s weighted sum would be:

(150∗0.5)+(0.9∗1.0)+1.5=76.4(150∗0.5)+(0.9∗1.0)+1.5=76.4

Let’s assume the activation function uses a threshold of 75. Since 76.4 > 75, the perceptron classifies the fruit as an apple (output = 1).

What Can Perceptron Do?

Despite their simplicity, perceptron can be used for binary classification tasks. Imagine you want to classify emails as spam or not spam. You could use features like the presence of certain keywords, the sender's address, etc., as inputs to a perceptron. The perceptron, after being trained, could then decide if a new email is likely to be spam (output 1) or not (output 0).

Other examples include:

Image recognition: Determining if an image contains a cat or a dog (for very simple cases with limited features).
Medical diagnosis: Predicting if a patient has a certain condition based on test results.

Algorithm:

Perceptron learn by adjusting their weights and bias for every input data. When a perceptron makes a wrong prediction, we need a way to quantify this error. This is where the concept of loss comes in.

For a single perceptron in a simple binary classification task, a basic loss function could be:

0 if the prediction is correct.
1 if the prediction is incorrect.

The goal of training is to minimize this loss over all the training data. The perceptron learning rule updates the weights based on the error:

new weight = old weight + learning_rate (expected output - predicted output) input

if the expected output and predicted output are different the straight line moves.

Limitations of Perceptron

The Perceptron was a significant breakthrough in the development of neural networks, proving that simple networks could learn to classify patterns. However, the Perceptron model has certain limitations that can make it unsuitable for some tasks:

Limited to linearly separable problems
Struggles with convergence when handling non-separable data
Requires labeled data for training
Sensitive to input scaling
Lacks hidden layers for complex decision-making

To overcome these limitations, more advanced neural network architectures, such as Multilayer Perceptron (MLPs) and Convolutional Neural Networks (CNNs), have been developed. These models can learn more complex patterns and are widely used in modern machine learning and deep learning applications.

We will discuss these models later, stay Tuned.

Perceptron model simplified