WTF is An LLM?

🚀 First Things First: What is an LLM?

An LLM — short for Large Language Model — is like a supercharged text-predicting machine. Think of it as a really clever parrot that’s read basically everything on the internet: books, blogs, code, tweets, memes, you name it.

But instead of just copying stuff, it learns patterns in language so it can respond to you like a human would. It doesn’t think like us — but it’s really good at guessing what word comes next.

We all have used ChatGPT have you ever wondered what is the full form of GPT

G - Generative - It generates text

P - Pre-trained - It’s already been trained on a huge amount of text before you even use it.

T - Transformer - This is the type of model architecture it’s based on

That’s the basic idea. Now, let’s talk about how it does that.

🧠 The Secret Sauce: Transformers

Behind every LLM is a magical model called a Transformer (no, not the robot kind).

Transformers changed the AI game. They let models look at all the words in a sentence at once, not just one at a time. That makes them insanely good at understanding context.

The paper that introduced Transformers was cheekily titled “Attention is All You Need.” Turns out that was kinda true.

✂️ Step 1: Tokenization — Chopping Up Language

Before the model can understand anything, it needs to break down your text into smaller chunks — called tokens. These are usually words or pieces of words.

For example:

“Unbelievable” → might turn into → "un", "believ", "able"

Each token gets a unique ID — like turning words into LEGO blocks the model can play with.

The total set of tokens it knows? That’s called its vocabulary size. The bigger the vocab, the more it can understand and generate.

import tiktoken
tokenizer = tiktoken.encoding_for_model("gpt-4")
text = "WTF is an LLM?"
tokenized_words = tokenizer.encode(text)
print(tokenized_words)

# output = [100264, 9125, 100266, 2675, 527, 264, 11190, 18328, 100265, 100264, 882, 100266, 54, 9112, 374, 459, 445, 11237, 30, 100265, 100264, 78191, 100266]

🔢 Step 2: Embeddings — Giving Tokens Meaning

Next up: turning those tokens into vectors — basically, lists of numbers that hold meaning.

Why? Because computers don’t understand text — they understand numbers.

These vectors are like coordinates in a giant meaning-map, where words like “cat” and “dog” are close together, but milk & pedigree is far away when cat is mapped to milk and cat and dot are clost to each other, with some calculations we can find the position of pedigree. It’s how the model gets a sense of what each word means.

🧭 Step 3: Positional Encoding — Remembering Word Order

Okay, so we’ve got tokens, and they’ve been turned into vectors. But here's a problem: computers don’t naturally know the order of words.

“Dog bites man” is not the same as “Man bites dog,” right?

That’s where positional encoding comes in. It adds a little bit of extra information to each token so the model knows which word came first, second, third, etc.

It’s like giving each word a number tag: “You’re in spot 1,” “You’re in spot 2,” and so on.

👀 Step 4: Self-Attention — What Words Matter Most?

Now the fun part: self-attention.

Here, each word looks at all the other words in the sentence and decides which ones it should pay attention to.

Like, in the sentence:

“The cat sat on the mat because it was tired.”

What does “it” refer to? Self-attention helps the model figure that out. “It” is probably talking about “the cat,” not “the mat.”

This ability to connect the dots across the sentence is what makes Transformers so powerful.

🧠 Step 5: Multi-Head Attention — Looking at Everything From Different Angles

Self-attention is great, but one perspective isn’t always enough.

So we use multi-head attention — which means the model looks at the sentence in multiple ways at once. Each “head” focuses on different relationships between words.

It’s like having a team of detectives, each noticing something different, then pooling their findings.

🧽 Step 6: Clean-Up & Processing — Normalization and Feedforward Layers

After attention, the model does some cleanup. It uses normalization to keep the data balanced, and a feedforward network (basically, a small neural net) to process the results further.

This part is more about fine-tuning — not as flashy, but still important.

🧠 Final Step: Output — Turning Thoughts Into Words

Now the model has thought through everything… but how does it speak?

It runs all the info through a linear layer (which maps it to possible words), and then uses something called softmax to pick the most likely next word.

It chooses one, then does the whole process again… and again… until it finishes your sentence.

🎉 And That’s a Wrap!

So that’s how LLMs work — at a high level, and without making your brain.exe stop working.

They take your words, break them down, understand them using a crazy-smart system of numbers and attention, then build a reply one token at a time.

And yeah — it’s kind of like magic. But now you know the trick. 😉