What is GPT?

Let’s imagine you’re at school and your teacher says, “Let’s draw a picture of a tree.” You know what a tree looks like, so you start drawing it with a pencil. The act of creating the picture is Generative — that’s the G in GPT.

You’re able to remember what a tree looks like because you’ve seen it before or drawn it before. That means your brain is Pretrained with that knowledge — that’s the P in GPT.

When you listened to what your teacher said and followed the instructions to draw, you were able to produce an output based on an input. That’s what a Transformer does — the T in GPT.

So, this is GPT ~ Generative Pretrained Transformer.