Explaining Embeddings in Machine Learning like I'm 10

Words are very powerful! To help people understand us, we use them to communicate our ideas and thoughts. The funny thing is, although humans understand words, AI models don't actually understand them. All they understand are numbers! Thus, we need to help them by converting words into numerical values for them to grasp. Imagine a situation whereby we assigned a random number to every word, For example:

• "Apple" = 5

• "Banana" = 12

• "Cat" = 8

Now, if we want to teach the model about a sentence like "I love apples," we could change it to numbers:

• "I love apples" = [1, 3, 5]

The model sees those numbers and knows, "Oh, 5 means 'apple'!"

However, it isn't really useful to merely provide random numbers. It sounds like we're communicating in a secret code that only we know. Thus, we use a more effective method of converting words into numbers that make sense to the model.

That's where embeddings come in! We use clever techniques to assign numbers to words based on their usage, as opposed to assigning random numbers. For instance, "apple" and "banana" may have similar numbers if they are often used in sentences together. Thus, much as humans pick up new words by seeing them in sentences, the model picks up new words by seeing the words that surround them.

We refer to these unique numbers as embeddings, and they allow the model to start understanding words and even predicting words that could appear in a phrase! Therefore, for a model to understand us and for us to teach it something new, we have to convert our words into numbers, which is the language the model understands.