Query ROUTING (logical & semantic concepts) + Graph Database

In Retrieval-Augmented Generation, success depends on more than just good data — it’s about choosing the smartest routes for storage, search, and retrieval.

Every step, from indexing information to delivering answers, can be optimized. Query Routing acts as the GPS that guides RAG systems toward smarter outcomes.

Why Routing ?

In Real World applications of RAG, we have Fragmented Data, ( Data from multiple sources and of multiple types), so its not Ideal to Store all data after Indexing at single Place either in same database or same collection inside a database. So we can use Routing techniques for storing and retrieving them to different collections . More Efficient , Accurate Responses , Faster Query processing time

Query Routing : It points you towards correct path .

That path, can be anything depend on its use-cases, u might use it to Retrieve data ( Correct Collection from which data should be fetched ) , Storing Data ( Correct Collection to which data should be stored ) , selecting LLM ( select which LLM can best answer user query ) etc.

Its of two types in RAG:

1. Logical Routing - ( Routing based on logic )

2. Semantic Routing - ( Routing based on Semantic meaning )

Semantic routing thinks about what you mean. Logical routing follows what rules say .

Logical Routing :

Here, we can have multiple databases or multiple collections inside one database, to store different categories of data available, like, we can create multiple collections based on types of data, image, pdf’s, web data, database etc. or we can create collections based on their topic/context at time of Indexing process in RAG ,

On receiving user prompt**, we can pass it to LLM to decide Route and it decides Based on “Defined Rules”, which collection should query go for fetching relevant context for response generation.**

Defined Rules: They could be anything Name, Description, Conditions, Pros , Cons, etc.

Code Example using above Flow:

// Logical routing function
function logicalRouter(userPrompt) {
  const collections = {
    financial: ["invoice", "payment", "budget", "revenue", "expense", "finance", "cost"],
    employee: ["employee", "hiring", "recruitment", "payroll", "salary", "leave", "attendance", "promotion"],
    feedback: ["feedback", "review", "complaint", "suggestion", "rating", "survey", "opinion"]
  };

  const promptLower = userPrompt.toLowerCase();

  for (const [collectionName, keywords] of Object.entries(collections)) {
    for (const keyword of keywords) {
      if (promptLower.includes(keyword)) {
        return collectionName;  
      }
    }
  }
  return "general"; 
}

// Example usage:
const userQuery = "Can you pull last quarter's revenue reports?";
const routedCollection = logicalRouter(userQuery);

console.log("Selected Collection:", routedCollection);

// Output: Selected Collection: financial

Semantic Routing :

Its for very Specific RAG use cases, where we have limited a RAG which can do limited small number of tasks only.

Here, we write system prompt for LLM here, with some variables inside, which changes according to Semantic meaning of user query,

On receiving user prompt, LLM decides variables based on semantic meaning , and constructs / transforms user query accordingly, and based on that our LLM finds out response.

Semantic Prompt Examples:

"You are a Semantic Router. Your task is to understand the user's query based on its meaning 
and decide the best route for it.

Carefully analyze the intent, topic, and context of the query, and select the most appropriate destination from the available options.

Focus only on the semantic meaning of the query, not just keywords. 
Respond only with the chosen destination."

"You are an intelligent Semantic Router. Based on the meaning and intent of the user's query, choose the most suitable route from the following options:

    storage → For queries about uploading, saving, organizing, or storing data/documents.

    retrieval → For queries about finding, searching, retrieving, or accessing stored information.

    model_selection → For queries that need deciding which LLM or model should answer (for example, technical vs. creative queries).

    summarization → For queries that involve condensing long information into shorter summaries.

    translation → For queries involving language conversion or multilingual content.

    fallback → For queries that don't match any above categories clearly.

Carefully understand the query's meaning, and respond with the most suitable variable name only."

"You are a Semantic Router.
Your task is to deeply understand the meaning and intent behind the user's query.
Fill in the following variables based on your understanding:

    {action} → What is the user trying to do? (e.g., store, retrieve, summarize, generate, translate, analyze)

    {topic} → What is the query about? (e.g., documents, coding, travel, marketing, data)

    {target_model} → Which type of model should handle it? (e.g., general LLM, code LLM, retrieval system, summarization model)

    {urgency} → Is it urgent, normal, or can be queued?]

Analyze the query carefully, rely on semantic meaning rather than keywords, and fill these fields accurately. If information is missing, infer sensibly."

Graph Database:

Although its not Routing part, but since its used in Routing , RAG and can be considered Routing part, i am explaining a bit here

We were using Vector Databases for vector embeddings of our Data sources till now, but

Vector Database - only good for storing “ Semantic Meaning “ , but they lack in storing Logical Relationships

to store Logical Relationships , we have Graph Databases

and their combination is Powerful Hybrid System

Combining Graph and Vector databases lets AI both "understand meaning" and "reason through relationships," making retrieval much smarter than either alone.

Both together = Powerful AI Reasoning + Accurate Answers.

Example with diagram :

Imagine you are building an AI system for a sports analytics platform.
You have two layers of data:

• Graph database layer (like Neo4j):
  Stores explicit relationships between players and teams.
  Example:

  • Rohit Sharma → Teammate of → Virat Kohli

  • Rohit Sharma → Plays for → Team India

  • Virat Kohli → Plays for → Team India

• Vector database layer (like Pinecone, Chroma, or FAISS):
  Stores semantic embeddings (vectors) of player profiles, team profiles, and articles about them.
  Example:

  • "Rohit Sharma" and "Virat Kohli" are nearby in vector space because they are both top-order Indian cricket players.

  • "Team India" is close to them because it represents the national cricket team they both play for.

Now, here's how combination would work in practice:

Suppose a user asks the LLM:

"Which Indian players who are close friends also play for the national team?"

1. Vector Search first:

   • Find players semantically related to "Indian players" and "close friends" in the vector database.

   • It identifies that Rohit Sharma and Virat Kohli are semantically close (their descriptions mention friendship, Indian team )

2. Graph Traversal second:

   • From those vector-retrieved players, the graph database is used to verify explicit relationships:

     • Check if there is a "Teammate" relationship.

     • Check if they both have a "Plays For" relation with "Team India".

3. Final Answer:

   • LLM returns:

     "Rohit Sharma and Virat Kohli are teammates and both play for Team India."

Summary :

• Routing points you towards correct path .

• Logical routing follows what rules say .

• Semantic routing thinks about what you mean.

• Vector DB gives approximate similar people/entities (fuzzy matching by meaning).

• Graph DB checks logical relationships between them (strict matching by relationships).

Conclusion:

Just Explained what i understood about Routing, types, Graph database & its Combo with vector database If you find it useful then don’t forget to like this Article & follow Me for more such informative Articles .

Credits:

Credits: I am very grateful to ChaiCode for Providing all this knowledge, Insights , Deep Learning about AI : Piyush Garg Hitesh Choudhary

If you want to learn too, you can Join here → Cohort || Apply from ChaiCode & Use NAKUL51937 to get 10% off

Thanks:

Feel free to Comment your thoughts, Would love to hear your feedback..

Thanks you for giving your Precious time , reading this article. Have a Great Day :)

Connect:

Let’s learn something Together: LinkedIn, Twitter

If you would like , you can Check out my Portfolio