Decoded AI Jargons

Transformers - It refers to a specific type of neural network architecture that excels at processing sequential data like text. Understanding relationships between elements, making them powerful for tasks like natural language processing (NLP) and beyond.

Encoder - It is responsible for transforming data into latent representations.

Decoder - It is responsible for transforming encoded data into meaningful output format.

Vectors - It shows the data with the magnitude(size)(length of the arrow) and direction.

Embeddings - It means a numerical representation of data.

Positional Encoding - It is used to provide a relative position for each token or word in a sequence. Without position info, attention would treat:

“The cat chased the mouse”

“The mouse chased the cat”

as potentially the same, because it only considers token relationships, not their positions.

That’s why positional encodings are crucial.

Semantic Meaning - It simply means context or understanding of something

For example -> If I told you “How are you”. The Computer is going to understand the same as we Human understand like “aap raise ho”, Computer will understand what we’re telling him.

Softmax - It is a function, commonly known as temperature in Chatgpt or Google AI Studio Playground. Which helps us to select the probability of the answer according to our current data. Low Temperature means high probability and High temperature means low probability.

Multi-head Attention - Single attention has query, key, and value. After multiple parallel attention, It will concat.

For more Info - check this video, it's an useful video using animation.

https://www.youtube.com/watch?v=mMa2PmYJlCo

Temperature - It’s a softmax function, which helps to select randomness.

Knowledge Cutoff - It represent the last date on which model is trained.

Tokenizations - It is the process of breaking down text into smaller units called tokens (words, subwords, or characters).

Vocab Size - The number of unique words or tokens a model is trained on.