APEX: Exploring AI Configurations

AI Configurations is a new feature introduced in version 24.2, centralising Generative AI settings in the applications shared components. It also allows for multiple custom Retrieval-Augmented Generation (RAG) or custom sources to be queried, combined and included in GenAI conversations.

Using RAG, text generated from GenAI conversations can be more relevant, current and based on data sourced from within the Enterprise.

About this Post

While working with a customer to implement the sample app and solution from the Integrating OCI Generative AI Agents with Oracle APEX Apps for RAG-powered conversational experience blog by Roopesh Thokala, a colleague had similar requirements but who wanted to leverage OOTB APEX AI dynamic action plugin.

The discussions lead to some interesting questions

• What is returned by an AI Configuration with multiple RAG Sources or different types?

• What are the best methods to include conditions and security filters on RAG sources?

• What APEX features can be leveraged with AI Configurations, Workflows, Automations, … ?

• What are the draw backs?

Time to check under the covers and explore our options

Getting Started …

AI Configurations start with basic settings: name, system prompt, welcome message, setting randomness temperature, and selecting the APEX AI Service.

This is the section where the artists meets the scientist and you can add the prompt engineering to constrain and direct the GenAI services response. There are sites, courses and guides recommending best practices on this topic, so if is new to you, use your GenAI services of course and ask for some references.

here are some links:

• OpenAI

• Google

• Microsoft

I just think of it as the tracks to keep GenAI going in the right direction.

Welcoming message and Guardrails in place, time to add the RAG sources.

Working with RAG Sources

APEX AI Configurations support three types of sources: SQL Query, Function Body returning CLOB, and Static text.

SQL Query

A standard select statement, supporting page & application bind variables

Use where clauses and index strategy to optimise performance, improving response times, as submitting large datasets to LLMs will slow down what can already be a slow process, and certainly increase the amount of tokens, equating to higher costs.

Function Body

Oh yeah, this is where it’s at!

All the flexibility your heart desires and in the choice of two languages, PL/SQL or javascript.

• Any code set that you would add as a function or procedure

• Invoke multiple sets of functions or services over Web Services, prepared before returning the result set(s)

Output the result in a CLOB

The main reasons I would use it, other than preparing the data, is to reduce and filter or remove anything sensitive that should not be shared.

Toufiq has written a post back in June → Building Retrieval Augmented Generation in Oracle APEX with AI Configurations and RAG Sources based on the getting started Build an Innovative Q&A Interface Powered by Generative AI lab.

The main consideration is the number of tokens to process the request once data is added to it.

Review the Bonus Tip for added Vector Search to reduce the data as much as possible.

Static Text

Use this for standard information that should be included in every AI request. This could include disclaimer, usage guidelines, or a standard response when server-side conditions for all other RAG sources are not met.

As this is APEX and Static, it may be possible to reference &SUBSTITION_STRINGS.

Q&As

At the beginning I shard these questions:

• What is returned by an AI Configuration with multiple RAG Sources or different types?

• What are the best methods to include conditions and security filters on RAG sources?

• What APEX features can be leveraged with AI Configurations, Workflows, Automations, … ?

• What are the draw backs?

Let me open up a considerations box for us to place ideas or concerns.

What is returned by an AI Configuration with multiple RAG Sources or different types?

The answer is simple, multiple sets of data that are included in the chat prompt. The type does not matter, CSV formatted Query results, CLOB, JSON, or static text, all of the processed RAG sources will be sent to the LLM.

Of the considerations or possible impacts is that data will be shared with the LLM, If it is not hosted locally, privately, what have you just shared outside of the enterprise?

What is the cost of sending large amounts of data, and can it be processed efficiently by the chat interface over REST?

Running tests with my colleagues we had varying results and the data sets were seemingly small, from the database’s point of view, but the results for the same question where not just wrong, it was different every time.

Were we asking to much of the LLM? I do not know but certainly a combination of both SQL and NLP + context with just enough data is the main goal.

Leverage as much of the database built-in tools before invoking your services and design a RAG strategy, similar to the first purpose built data warehouses which had only the amount of data to answer a predefined set of questions.

What are the best methods to include conditions and security filters on RAG sources?

Honestly, I have not used this enough to put forward a “best” of anything. What I do know is that using these services cost money, processing power, and can impact an applications user experience.

The server side conditions control the execution of the RAG code. Supporting text in the user’s prompts, APEX expressions, functions and more.

For example if the user has not asked about policies, then do not execute the code that call out to the GenAI agent that searches policy documents.

Authorization Schemes are an essential APEX security practice allowing access to components based on user roles. So when the user is not in role HR, they will not get answers to broader employee questions that are not self-service related.

Though not available as a RAG source property, I strongly recommend the use of the Where Clause in all queries and functions. The reasons should be apparent and with time we will get better at stripping down the prompt, extracting relevant terms that we can also pass into the where clause.

What APEX features can be leveraged with AI Configurations, Workflows, Automations, … ?

OOTB only two AI-enabled components such as Show AI Assistant and Generate Text With AI dynamic actions are linked to AI Configurations.

For everything else, there is the APEX_AI package where the Static ID is used to reference the AI Configuration in the APEX_AI package, allowing them to be used in any place we can place a PL/SQL Function Body.

What are the draw backs?

Security & Risk, and Performance & Cost

We will all need to understand the implications of sharing data. Once outside of the enterprise. Are LLM providers bound by data protection rules when we share this information so freely?

Isabel Barberá’s published paper: AI Privacy Risks & Mitigations provides some insight into the topic.

Looking at performance and costs, less data tends to be better. Sharing less might sound great but at what point does it affect the quality of the response? On the flip side, how much is enough and when including more no longer yields better responses.

Processing less data should allow for faster responses but like everything else, it depends. GenAI is truly ART + Science

Finding the right balance at the right price point is as challenging as using the services themselves. I quick search on the internet will yield many articles and services dedicated to the Cost Economics of GenAI.

Conclusion

AI Configurations are a great way to share and standardise the use of GenAI prompts and RAG Sources in APEX. Is it still LowCode, that is debatable.

RAG Sources have great benefits providing better context for LLMs to reference for their responses but should be used with proper planning and budgeting.

I encourage you to leverage all filtering and security controls as you can. I myself will never truly understand the wide ranging impacts it will have, positive or not but querying & submitting too much data can be just as bad as not sending anything at all, from a performance and cost point of view.

In the case of RAG responses, smaller is better, so tailor this as best as you can.