AI Answer Capabilities with ChatGPT and Elasticsearch
Shubham Shah
Overview
Learn how to build an AI Answer interface for your own domain with this follow-along post - where we use ElasticSearch and ChatGPT together to build an AI Answering system with high precision (no hallucinations), is always based on facts from your domain (instead of outdated facts it may have) and leverages Elasticsearch as a search engine effectively to filter and show results along-side the AI Answer.
To keep the user interface configurable for your use-case, we will use the ReactiveSearch UI kit's (Apache 2.0 licensed) AIAnswer component here. This is available for both React and Vue today.
LTM - QA
ChatGPT is a GQA (Generative Question Answering) system. The most straightforward GQA system requires nothing more than a user text query and a large language model (LLM).
We enhance GQA with LTM-QA, which stands for Long-Term Memory Question Answering. LTM-QA enables machines to store information for an extended period and recall it as needed. This improves both precision and recall - so a win-win.
The novel idea here is that we make use of BM-25 (TF/IDF) relevance instead of requiring vectorization of data, although this is also possible - a post on the same is coming soon. Elasticsearch and OpenSearch (which is the Apache 2.0 derivative of the former) are the search engines to beat when it comes to relevant text search with filtering capabilities. If your search is used by users who know the data well, then BM-25 based relevance offers instant results at scale and query remixing capabilities that are very nascent with vector search today.
The simple idea is to combine the power of ChatGPT and standard search engine tools like ElasticSearch and OpenSearch to finally come up with a powerful search experience that's relevant, accurate, to the point, and evolving as per the users' prompts.
🤔 Elasticsearch and OpenSearch are the standards (10x adoption, work at scale, and have good Dx) where developers already likely have their long-term search data, also they both support BM-25 and vector search.
ReactiveSearch is the CloudFlare for Search Engines
ReactiveSearch.io provides a supercharged search experience for creating the most demanding app search experiences with a no-code UI builder and search relevance control plane, AI Answering support (what we will be using over here), pipelines to create an extendible search backend, caching and search insights.
There are additional open-source and hosted tools that simplify the process of building search experiences. You can:
Import your data from various sources via the dashboard or CLI or REST APIs,
Build production-grade search UIs using:
UI component libraries that are available for React, Vue, React Native, Flutter, and vanilla JavaScript,
Get out-of-the-box actionable analytics on top searches, no-result searches, slow queries, and more,
Get improved search performance and throughput with application layer caching,
Build access-controlled search experiences with built-in Basic Auth or JWT-based authentication, read/write access keys with granular ACLs, field level security, IP-based rate limits, and time to live - read more over here.
Dataset and relevant questions
To make a great UI we should have a good dataset. We would be using a dataset of dialogues from the popular TV Show "Rick and Morty". We would also need an index to store the data.
You can set up and install an Elasticsearch server by following the official installation guide, or you can create a free account at reactivesearch.io which provides Elasticsearch hosting as a service and is easy to use. For simplicity, we will be using reactivesearch.io service to get started.
I’ve already created an index with the dataset. You can check out the dataset from above over here in the data browser tool Dejavu, which is built by reactivesearch.io.
We also have a long list of questions in JSON format with the following structure. Each question is linked to an episode, which is useful for finding documents in the above index for a particular episode.
[
{
"question": "How many Ricks and Mortys are there?",
"episode": "A Rickle in Time"
},
{
"question": "What's the most bizarre time paradox Rick and Morty encounter?",
"episode": "A Rickle in Time"
},
{
"question": "What's the funniest time-related mishap?",
"episode": "A Rickle in Time"
},
]
You can also have a separate index for the questions if you have a large list which makes the search slow.
Building Search UI
All the code we are going to show is present inside the qna-rick-and-morty repository. We also recommend going through the quick start guide of the Reactivesearch UI library to become familiar with it. The guide will not only show you how to set up a boilerplate React app, but also how to set up the @appbaseio/reactivesearch library. We will primarily modify the src/App.jsx file, which will also introduce you to ReactiveBase and SearchBox component that we will be using in this tutorial. Additionally, we will introduce a new component called AIAnswer, which creates a ChatGPT-like search experience.
Configuring ReactiveBase:
We would want to connect to rick-and-morty index. For that we configure ReactiveBase component as follows:
// src/App.jsx
function Main() {
return (
<ReactiveBase
app="reactivesearch_docs_v2"
url="https://a03a1cb71321:75b6603d-9456-4a5a-af6b-a487b309eb61@appbase-demo-ansible-abxiydt-arc.searchbase.io"
theme={{
typography: {
fontFamily: "monospace",
fontSize: "16px",
},
}}
reactivesearchAPIConfig={{
recordAnalytics: false,
userId: "jon",
}}
>
{/*All components go here*/}
</ReactiveBase>
);
}
const App = () => <Main />;
export default App;
Adding SearchBox:
We would make a SearchBox component with the following config. You would be familiar with the props from the quick start guide. All the props are also listed in the documentation.
function Main() {
return (
<ReactiveBase
{...config}
>
<SearchBox
dataField={["episode_name", "name", "line"]}
componentId={SEARCH_COMPONENT_ID}
size={5}
autosuggest={true}
/>
</ReactiveBase>
);
}
To search through the list of questions and show suggestions when the user types, we will be using a library called fuse.js. Fuse.js is a lightweight fuzzy search library that is appropriate for the size of our data. If the data were larger, we would have created an index for better performance.
You need to install it first. The latest version at the time of writing is v6.6.2.
yarn add fuse.js
You can look at the documentation for Fuse.js though the usage is quite simple. You just have to pass data to Fuse instance and then call the search method passing your searchQuery. Note that we append the episode name in the question itself which would be used to get the documents filtered by the episode.
// src/App.js
// questions: [{question, episode}]
const fuse = new Fuse(
questions.map((q) => ({
...q,
question: q.episode
? `${q.question.replace(/\?$/, "")} in "${q.episode}"?`
: q.question,
})),
{ keys: ["question"] }
);
const filteredData = fuse.search(searchQuery).map((res) => ({
value: res.item.question,
episode: res.item.episode,
idx: res.refIndex,
}));
We would then render this filteredData using the custom render.
function App() {
return (
<ReactiveBase {...configProps}>
<SearchBox
{...previousProps}
render={({
downshiftProps: {
isOpen,
getItemProps,
highlightedIndex,
selectedItem,
},
data,
value: searchQuery,
}) => {
const fuse = new Fuse(
questions.map((q) => ({
...q,
question: q.episode
? `${q.question.replace(/\?$/, "")} in "${q.episode}"?`
: q.question,
})),
{ keys: ["question"] }
);
const filteredData = fuse.search(searchQuery).map((res) => ({
value: res.item.question,
episode: res.item.episode,
idx: res.refIndex,
}));
return isOpen ? (
<div className={`${styles.suggestions}`}>
<div>
{filteredData.length ? (
<p className={`bg-gray p-2 m-0 ${styles.suggestionHeading}`}>
Try below suggestions
</p>
) : null}
{filteredData.length ? (
<div>
{filteredData.map((item, index) =>
index < 6 ? (
<div
/* eslint-disable-next-line react/no-array-index-key */
key={item.idx}
{...getItemProps({
item,
})}
className={`${
highlightedIndex === index
? styles.activeSuggestion
: styles.suggestion
}
${
selectedItem &&
selectedItem.value === item.value
? styles.selectedSuggestion
: ""
}
`}
>
<span className="clipText">{item.value}</span>
</div>
) : null
)}
</div>
) : null}
</div>
</div>
) : null;
}}
/>
</ReactiveBase>
);
}
Adding the AIAnswer Component
AIAnswer is an AI-driven answer UI component that interacts with a dataset to provide context-aware and relevant answers based on user inputs. It employs machine learning to comprehend user questions and retrieve the most pertinent information. The component can be used to supply answers to common questions related to a specific topic, offer support and help by addressing user queries, and create a knowledge base or FAQ section for your website.
Learn more about the AIAnswer component over here.
The AIAnswer component adheres to the same principles as other ReactiveSearch components. It has a react prop that lists dependencies. We will add the SearchBox component as a dependency so that when we hit Enter and the SearchBox triggers a query, the AIAnswer component will display the relevant answer.
There is an additional prop called AIConfig, which allows configuring the default prompt and parameters sent to ChatGPT. Referring to the diagram in the "Let's Visualize" section, we feed documents from Elasticsearch to ChatGPT. These documents are the top results found by performing a search (the suggestions seen in the SearchBox component). The format is determined by docTemplate prop, where we can specify any field from the search index (rick-and-morty in this case, this would vary based on your index). topDocsForContext dictates how many documents to feed to ChatGPT. queryTemplate is the format for the actual question fed to ChatGPT after the document queries, which is formed from the value provided in the SearchBox.
// src/App.js
function App() {
return (
<ReactiveBase
{...configProps}
>
<SearchBox
componentId={SEARCH_COMPONENT_ID}
{...otherProps}
/>
<div id="answer-component" className={`px-5 pt-5 ${styles.answer}`}>
<AIAnswer
componentId={AI_ANSWER_COMPONENT_ID}
showVoiceInput
showIcon
react={{ and: SEARCH_COMPONENT_ID }}
AIConfig={{
docTemplate: "'${source.name}', says '${source.line}'",
queryTemplate: "${value}",
topDocsForContext: 15,
}}
enterButton={true}
showInput={false}
/>
</div>
</ReactiveBase>
);
}
In our case, we also want to show the documents that were used to generate the AI answer in the UI itself, so we know the answers come from the search index and the AI isn't hallucinating or replying from out-of-domain facts. We store the data in the state using the onData handler.
function App() {
const [feedData, setFeedData] = useState({ loading: true, hits: [] });
return (
<ReactiveBase
{...configProps}
>
<SearchBox
componentId={SEARCH_COMPONENT_ID}
{...otherProps}
/>
<div id="answer-component" className={`px-5 pt-5 ${styles.answer}`}>
<AIAnswer
componentId={AI_ANSWER_COMPONENT_ID}
onData={({ loading, rawData, data }) => {
if (!loading && data.length) {
if (rawData.hits && rawData.hits.hits)
setFeedData({
loading: false,
hits: rawData.hits.hits.map((hit) => hit._source),
});
} else {
setFeedData({ loading: true, hits: [] });
}
}}
{...otherProps}
/>
</div>
</ReactiveBase>
);
}
Getting episode from the search query
Since we store the associated episode for each question suggestion, we would like to use this information in getting the first-pass results from the search engine. In a previous step, we put the episode name in the question, and now we will extract it and tell the index to only show those documents which match the episode name.
We will use the transformRequest method to do this, which lets us modify the payload generated by the ReactiveSearch library before sending it across the network.
function App() {
const [feedData, setFeedData] = useState({ loading: true, hits: [] });
return (
<ReactiveBase
{...configProps}
transformRequest={(req) => {
const body = JSON.parse(req.body);
body.query = body.query.map((componentQuery) => {
if (
componentQuery.id === SEARCH_COMPONENT_ID &&
componentQuery.type === "search"
) {
const searchQuery = componentQuery.value;
const matches = searchQuery.match(/in "(.*?)"\?$/);
const episode = matches && matches[1];
return {
...componentQuery,
customQuery: {
query: {
term: {
"episode_name.keyword": episode,
},
},
},
};
}
return componentQuery;
});
body.settings = {
recordAnalytics: false,
backend: "opensearch",
};
const newReq = { ...req, body: JSON.stringify(body) };
return newReq;
}}
>
</ReactiveBase>
);
}
The complete app should look as below:
Summary
Hope you enjoyed reading! Here's a brief recap of what we covered, so you can implement the same for a domain of your choice:
ChatGPT is a GQA system with limitations, which can be overcome by using LTM-QA to improve both precision and recall.
We used a search index of Rick and Morty dialogues to retrieve documents from when user performs a search.
We utilized the
SearchBoxcomponent to suggest a list of popular questions that a user might have.We added an
AIAnswercomponent that depended on the SearchBox and updated dynamically based on the user's query.Lastly, we filtered documents relevant to the query (specific episode) to improve the precision of the answer.
Subscribe to my newsletter
Read articles from Shubham Shah directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by