Chain of Thought: Understand the user's Query

Now that you've learned how to make AI step back and think, let's go one level deeper. The next cool technique is called Chain-of-Thought Prompting.

What Is Chain-of-Thought Prompting?

Imagine solving a math problem. You don’t just say the answer—you show your steps. That’s exactly what Chain-of-Thought (COT) Prompting makes the AI do.

Instead of just giving a final answer, you ask the AI to explain its reasoning step-by-step.

Without Chain of Thought vs. With Chain of Thought

Question:
John has 7 apples. He buys 3 more baskets of apples. Each basket contains 4 apples.
How many apples does he have now?

Answer: Without Chain of Thought
AI: 19 apples.

The answer is correct, but the reasoning is opaque. It’s unclear if the model calculated properly or made a lucky guess.

With Chain of Thought
AI: John starts with 7 apples.
He buys 3 baskets, and each basket contains 4 apples.
So, 3 × 4 = 12 new apples.
Now, 7 + 12 = 19 apples in total.

The answer remains correct, but the step-by-step breakdown clarifies how the conclusion was reached. This transparency is critical for troubleshooting errors or explaining decisions in high-stakes scenarios.

Key Difference:

• Without CoT: Answers feel abrupt and untrustworthy for complex tasks.

• With CoT: Reasoning is explicit, fostering confidence and enabling users to validate logic.

Code Example for Chain-of-Thought Prompting

from google import genai
from openai import OpenAI
from dotenv import load_dotenv
import os
load_dotenv()

system_prompt = """
You are a helpful assistant that thinks step by step.
Example-
Alex have 12 banana, he give 2 to John 3 to Jyuli haw much now Alex have,
do it step by step 
1. Alex have 12 Banana
2. He give 2 to John (12-2)=10
3. He give 3 to Jyuli (10-3)=7
4. Alex have 7 Banana have
"""


client = OpenAI(
    api_key=os.getenv("API_KEY"),
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/"
)

response = client.chat.completions.create(
    model="gemini-2.0-flash",
    n=1,
    messages=[
        {"role": "system", "content": system_prompt},
        {
            "role": "user",
            "content": "Ram have 100 rupees he give 10 to rahul 20 to rohan 15 to hitesh how much ram left"
        }
    ]
)

print(response.choices[0].message.content)

'''Input -
Ram have 100 rupees he give 10 to rahul 20 to rohan 15 to hitesh how much ram left
 Output -
1.  Ram starts with 100 rupees.
2.  He gives 10 rupees to Rahul (100 - 10 = 90).
3.  He gives 20 rupees to Rohan (90 - 20 = 70).
4.  He gives 15 rupees to Hitesh (70 - 15 = 55).

So, Ram is left with 55 rupees.
'''

Summary

Chain-of-Thought (CoT) prompting is a technique used to guide AI models to think and reason step-by-step before giving a final answer. Instead of jumping straight to the solution, the AI is encouraged to break down the problem into smaller logical steps, just like how a human might solve it on paper. This approach leads to more accurate, transparent, and easy-to-understand responses, especially for complex questions involving reasoning, math, logic, or multi-step tasks. CoT prompting not only improves the quality of answers but also helps users follow and trust the AI’s thinking process.

Links of RAG-related Blogs

1. What is RAG

2. Parallel Query Retrieval (Fan Out)

3. Step back Prompting

4. Reciprocal Rank Fusion

5. Hypothetical Document Embeddings