For the last year or two, I took a short sabbatical from writing to focus on personal well-being and recalibrate other aspects of life. And what a time it has been to pause! In the blink of an eye, we’ve ushered in the era of agentic machines—where large language models have been democratised and made accessible at our fingertips.

As I began to explore the inner workings of generative AI, I was transported back to my early days of coding typewritten chatbots during my bachelor’s degree using .NET. Back then, I would manually feed a chatbot with a list of keywords and predefined responses. ChatGPT initially felt similar—except this version is exponentially more scalable and powered by recent machine learning breakthroughs. Instead of matching keywords to canned replies, it can predict and generate responses to entirely new prompts. It’s like a search engine on steroids—trained on vast data from the internet, capable of identifying patterns, predicting outcomes, and even creating original content.

While generative AI might seem like a recent phenomenon, it’s actually been decades in the making.

As early as 1950, Alan Turing posed the question, Can machines think? He introduced the now-famous Turing Test (TT)—where a judge interacts with both a human and a machine in two separate rooms. If the judge can’t tell which is which, the machine is considered to have passed the test.

This foundational idea gave rise to early AI experiments, including rule-based chatbots. These systems, however, had limited capabilities—responses were only triggered by exact keyword matches, making them impractical for complex or everyday problem-solving. The chatbots didn’t have any semblance of context or an understanding of the input text - they simply responded to whatever keywords they found in the input text.

In the 1990s, the rise of machine learning (ML) brought a statistical approach to language. ML algorithms could now recognize patterns and predict outcomes by analyzing large sets of labeled data. They simulated aspects of human understanding by classifying text and deriving context. However, limitations in hardware made it difficult to scale these models to handle massive datasets.

Enter the 2000s: advancements in hardware enabled the rise of neural networks, especially deep learning models. Recurrent Neural Networks (RNNs) became popular for tasks like natural language processing. Virtual assistants began to understand the context of conversations and respond using predefined scripts or APIs—for example, fetching the weather from a third-party service.

Of course, RNN came with their challenges:

• Vanishing and Exploding Gradients

• Short-Term Memory

• Sequential Computation

• Difficulty handling long sequences

In 2017, the introduction of the Transformer architecture revolutionized the field. It brought in a self-attention mechanism that eliminated the need for sequence-based processing and enabled parallel computation—greatly improving training speed and efficiency.

Today’s generative AI models, such as GPT (Generative Pre-trained Transformer), are built on this architecture. Transformers enhanced machines’ ability to not only understand input text but also generate rich, context-aware, and human-like language. These models are trained on vast amounts of unlabeled data and are capable of adapting to a wide range of tasks.

We’ll delve deeper into RNNs and the transformative model architecture in the next blog post, exploring how large language models actually work. But for now, here’s a simple analogy to help digest the evolution of Generative AI:

• Cooking (AI) → The broadest concept, encompassing all techniques for preparing food

• Baking (ML) → A more structured subset with specific ingredients and methods

• Bread Making (DL) → A refined skill requiring technique and fermentation

• Sourdough Art (GenAI) → The pinnacle of creativity—where bakers design beautiful patterns in bread

Thank you for reading—let's connect!

Enjoy my blog? For more such awesome blog articles - follow, subscribe, and let's connect.