Ever wondered how machines actually learn once they have been given the input data?
In yesterday’s post, we uncovered the foundational types of machine learning — supervised, unsupervised, semi-supervised, and reinforcement learning. But learning doesn’t stop there.

Just like not every student adopts the same method for learning , everyone has got different method for studying -some study from the start and are revising the already learned part, while there are students as well who cram the whole study in one night , Machine learning has got varied learning strategies as well. Some learn all at once, others adapt as new data flows in. Some memorize past examples, others build general models.

Batch Learning

Imagine you're preparing for an important exam. You don’t study every day, but instead, wait until you have all the chapters, notes, and questions ready. Then, you study everything in one go before the exam.
This is exactly how Batch Learning works in machine learning.

In batch learning, a machine learning model is trained on the entire dataset at once, rather than updating itself everytime it recieves new data. Once the model is trained it remains intact and does not learn from the data anymore. If we want to incorporate any change then the change should be included along with the entire dataset and then rather training for the changed part ,the model needs to be trained again from scratch.

Let us say you want to create a model for predicting the car price for a second hand car. You’ve collected a data of 1000 cars with features like: type of model, year of manufacturing, kilometers driven, mileage, colour, etc. In batch learning this data of 1000 cars is given into the machine learning system and a model is trained. It studies the entire data, identifies patterns like more the kilometers driven lesser the price and so on, and builds a complete model. Now suppose a new type of car arrives, then your model would not learn anything from this car example unless you retrain the model with previous 1000 cars. This proves to be a time consuming process and would require large amount of computational resources.

Batch Learning is useful in places where the data is static and does not change much. It can be used when you want high accuracy and stability from the model. Batch Learning is also called “offline learning” and is used in systems like fraud detection, weather forecasting where models are updated occasionally.

It is not suitable for applications which require real time learning , let us say in stock market prediction where the data changes frequently. In batch learning retraining the model again is expensive in terms of computational resources and is a time consuming process.

Batch Learning is like training a student using the full textbook at once. Once trained, the student doesn’t change their answers unless they study everything again with the new material.

Online Learning

Now imagine a student who does not wait for the whole syllabus. Instead, they study one chapter a day, and after each chapter, they update their notes and understanding. The next day, they move on to the next chapter, enhancing their learning as they move forward. That’s exactly how online learning works in machine learning.

Instead of waiting for the whole data , the model learns from the data in small batches updating itself continuously. It is ideal for real time systems or when data keeps coming in like instagram feed, stock prices.

Let us say you are building a machine learning system to predict whether a company’s stock price will go up or down. Stock markets generate new data every second like opening price, closing price, volume, 7 day average,etc. With Online Learning, your model can update itself as new stock data comes in throughout the day. Storing all historical stock data and retraining again from scratch would be time consuming and computationally expensive so instead, it learns continuously adapting to the latest trends. This is essential in finance where being late can cause loss of millions.

Online Learning is used in Recommender systems like Netflix or YouTube, Financial trading models, Live translation or speech recognition tools where the applications require real time learning and should be memory efficient which means it need not store all the historic data. Online Learning system is verstaile meaning it can adjust to new trends.

However in this learning style models can become unstable if the incoming data is noisy or inconsistent. It requires careful tuning to balance old and new knowledge so the model does not forget the earllier learnings. Also in this mistakes need to be handled carefully as mistakes made in earlier updates may carry forward if not corrected.

Online Learning is like a student who learns one page at a time who is continously updating his knowledge and staying in sync with new pages he learns.

Instance Based Learning

Let us say when we have university exams and we have very limited time to prepare, one approach is to go through all the previous year questions for preparation as there are very much high chances of questions getting repeated and also gives you a nice overview that how questions are being asked and from which topic. In exam if you encounter a new question than you try to recall the similar question from the memorised questions and try to guess the answer based on that. This is how exactly instance based learning works.

In this type of learning the model tries to memorize the training data and when it faces a new input, it tries to find similar past instances which it has memorized and makes predictions based on those. It is very simple to implement and works amazingly well with small and clean datasets. It is extremely useful in cases where the data is limited.

Instance-Based Learning like K-Nearest Neighbors , is useful in real world scenarios where comparing new data to past examples is more effective than building complex models. In medical science, Doctors often compare a patient’s symptoms to past cases. Similarly, KNN or other instance-based models can match a new patient’s symptoms, test results, or images with historical medical cases to suggest a likely diagnosis. When estimating the price of a house, instance-based learning can find similar houses nearby (same location, area, age) and average their prices. This method works well in housing markets where local context matters.

The major disadvantage of instance based learning is that it takes a lot of time in predicting. It is slow in predicting as it compares with all the past data. Since it compares with all the past data it is evident that it would require large amount of memory to store all the past data in order to process it again. This type of learning can be sensitive to noise and irrelevant features, so proper feature selection should be done and the model should be wisely tuned. The performance of the model should be closely monitored.

Instance-Based Learning is like a student with a giant memory book. It is great for exact matching problems.

Model Based Learning

Imagine a student who is preparing for the upcoming university exam. He is a type of student who studies the subject hard, understands the concepts properly. He won’t just cram the concepts or learn them by heart instead he builds a mental map of how things work. In exam, he would apply his understandings and learnings to solve the new question. That’s exactly how model based learning works.

In Model Based Learning, it studies the training data and builds a mathematical model out of that. Once trained this mathematical model can be used to make predictions on new data without referrring to the past examples. It is extremely useful with large datasets as storing the large datasets and remembering them would require large amount of memory resources.

This type of learning is used in situations where you need a system that can generalise from the training data and can make fast predictions. Machine learning models like Linear Regression or deep learning models like LSTM can learn from historical stock data and build a predictive model. Once trained, the model can predict future trends without rechecking old data every time. Self-driving systems use model-based reinforcement learning to predict the outcomes of different actions and decide which path is safest or most efficient. These models help plan turns, avoid collisions, and follow rules by generalizing training data. Companies use historical sales data to train models like Random Forests or Gradient Boosting. The trained model can then predict next month's sales based on factors like season, past trends, etc.

The major disadvantage of this type of learning is that it takes huge amount of time to build a mathematical model. Also it won’t perform well if the model which you selected does not fit the data. There are chances of overfitting or underfitting the data, so the system should be properly tuned and monitored closely.

Model-Based Learning is like a student who understands the logic. It is efficient and powerful where generalization is needed

Challenges in Machine Learning

Now that we’ve explored the different learning styles in machine learning, it’s important to understand that just like students face challenges like tough topics, confusing questions, or time pressure during exams, machine learning models also struggle but in their own ways. These struggles can affect how accurate, fair, or efficient a model can turn out to be. Behind every successful ML system are numerous obstacles that data scientists have to overcome. Let’s walk through some of the common struggles in the ML world.

1. Limited Availability of Training Data

   Machine Learning Algorithms take large amount of data to work properly. Even for simple problems, you might need thousands of examples for optimum results. For complex problems , this number might even reach to millions. In many cases, the training data is expensive to collect and simply not available.

   For example - A startup is trying to build a facial recognition model and have only 200 images, which is far too less to capture real-world variations.

2. Poor quality of Training Data

   If your training data is full of errors, outliers, and noise which may be possible due to poor or faulty measurement instrument, it will make it harder for the system to train the model perfectly, so your system is not likely to find optimum results. That’s why the first step is always data cleaning where the faults and errors in the data are removed. The area where most of the data scientist spend significant amount of time is in data cleaning.

   For example - In a disease prediction model, if patient records have missing symptoms or wrong labels, the model will wrongly diagnose the new patients.

3. Bias in Data or Model

   If your training data is biased, your model will be too. This can lead to unfair or inaccurate predictions, especially in sensitive applications like lending or healthcare.

   For example- For a waste classification model, if your dataset has more than half images of the total images for just 1 class ( lets say battery ), then there are high chances the model would be biased towards that class only, leading to inaccurate results.

4. Overfitting: Learning too much

   Overfitting occurs when the model learns the training data too well, including its noise and outliers, and fails to generalize to new, unseen data.

   For example- Imagine a student in your class, who performs very well in the college tests because the questions were exactly same as that were discussed during the lectures. He performs very miserable in university exams as there were new questions , similar to those discussed in lectures , but not the exact ones.

5. Underfitting: Learning too less

   On the flip side of overfitting, underfitting happens when the model is too simple to capture the patterns in the data. It neither performs well on training data nor on new data.

   For example- Imagine that student in your class, who neither performs well in the college tests nor in the university exams because he has not grasped the core concepts at all. No matter how the questions are framed, he is not able to answer them.

6. Dynamic Environments

   Models trained on past data may not adapt well when the environment changes. This is known as concept drift.

   For example- A fraud detection model trained on last year’s data may miss new fraud patterns , unless it is updated regularly.

7. High Computational Cost

   Training large models requires powerful hardware, time, and energy, which might not be possible for everyone.

   For example- Training a large language model like ChatGPT or Gemini would take weeks and millions of dollars , which is something a small research lab cannot afford.

---

Today, we looked at how machines learn and what makes that process challenging. Understanding this theory will help you make better decisions when building real models. Just like students prepare for exams using different study methods, machine learning models also learn in their own unique ways and each method has its strengths and challenges. Learning machine learning is a bit like learning how to ride a bike you need to understand the theory, but you need to practice it. From tomorrow, we’ll start putting things into action starting with Python, the most beginner-friendly language used in ML and I will show you how to set it up on your computer in minutes for the exciting work ahead.