Decoding AI Jargons With Chai

Introduction to AI

• OpenAI's earliest model is GPT, which stands for Generative Pre-trained Transformer. It doesn’t generate pre-trained content, but rather it is pre-trained on vast text data and then generates human-like text based on that training. This architecture is inspired by the Transformer model introduced by Google in their groundbreaking paper, “Attention is All You Need”, which also laid the foundation for advancements like Google Translate. What started as a tool for better language understanding is now evolving into systems that can assist and sometimes even challenge the role of computer engineers.

Tokenization

• Tokenization is the process of breaking down human language into smaller units called tokens which are then converted into numerical representations. This step is essential because machines can't understand text directly. They process numbers. Tokenization makes it easier for AI models to perform operations like vector embedding and positional encoding, which help the model understand both the meaning of the words and the order in which they appear.

      import tiktoken
  
      encoder = tiktoken.encoding_for_modal("gpt-4o")
  
      print("Vocab Size: ", encode.n_vocab) # Vocab Size: 200019
  

Encode- Decode

• Encoding the message into tokens.

•   text = "Hii! How are you?"
  
    tokens = encoder.encode(text)
  
    print("Tokens: ", tokens) # Tokens: [343, 656, 765, 675]
  

• Decoding the message

•   text = encoder.decode(tokens)
  
    print("Text: ", text) # Text: Hii! How are you?
  

Vector Embedding & Semantic meaning

• Once a text is tokenized and encoded, each token is converted into a vector a list of numbers that captures certain characteristics or patterns of that word. This is known as vector embedding.

  These vectors help the machine understand semantic meaning that is, what a word means in a particular context, not just how it's spelled. For example, words like mother and father have similar vector directions, while mother and car will be far apart.

• Just like a human thinks about the meaning behind a question and connects it with context, a machine uses these vector embeddings to measure similarity and find the most relevant information.

• 

Positional encoding & Self Attention

• Machines don’t naturally understand word order. Positional encoding adds information about each word’s position so the model can grasp sentence structure.

• Self-attention lets the model focus on the most relevant words in a sentence even if the same word appears multiple times. It helps GPT build context and meaning by comparing how words relate to each other.

• Together, they allow the model to understand language more deeply and predict the next word more accurately.

That’s it for this blog!
If you found it helpful, don’t forget to like, share, and follow for more simplified takes on AI…

AI doesn't have to be confusing, just need to decoded properly. ✨✨