Note this blog builds on concepts found in the OCI 2024 Generative AI Professional Certification and Course. I added additional steps to resolve issues I encountered following the guides.

This course is fantastic & Oracle offer a free certificationpromotion through July 31, 2024.

The course describes how to use IDEs to build LLM applications through their Github Repository. This blog explains configuration of PyCharm with the examples, whilst my other blog explains how to set up Jupyter notebooks.

Note on WSL: I had to rework this blog as I wanted to use PyCharm on my WSL as I already had Python setup there for Web Scraping. However it became apparent that the PyCharm will only configure a WSL Interpreter when using the PyCharm Professional version. Therefore this blog is for PyCharm Community Edition.

Note on OCI Regions: According to here. At time of writing, Text Generational Models are only available in US Midwest (Chicago). These code examples is this blog will only work in US Midwest (Chicago). My region is Germany Central (Frankfurt) and only supports Chat - I have, however, adapted the scripts somewhat to Chat Models (see the final section). If you want to read more about this, I recommend reading this discussion.

This guide is for Windows, you'll need to adapt this for other OS's

Setup OCI key

Window Key + R% and type %USERPROFILE%
Create an .oci folder
In OCI, Click Profile > My Profile
Click API Keys > Add API Key > Download API Key
Don't touch anything in OCI yet
Copy the file to %USERPROFILE%\.oci
Back in OCI click Add
Don't touch anything in OCI yet, keep the window open
Copy the info from OCI to clipboard
Save the contents of the clipboard to a new file called %USERPROFILE%\.oci\config
Change the last line to (change as necessary)
```
key_file=c:\full\path\to\folder\.oci\oci_api_key.pem
```
My folder looks like this

Setup PyCharm

Download and install Visual C++ Build tools
Modify the installation by selecting this workload
Install Python. In Windows, drop into a dos-box and type
```
 python
```
Download & Install PyCharm Community Edition
Run PyCharm
Optional/Recommended: Dracula Theme: Go to Plugin Marketplace, and search Dracula, click Install
Create a new Project you can tick Welcome Script
Test Python by running the main.py script using the green play/run link in the top middle of the picture. It will say Hi PyCharm in the output window.
Next to the run button, it says main, change this to Current File
Click Terminal

Paste in the following to install the dependencies in the terminal box

pip install oci
pip install oracle-ads
pip install langchain
pip install chromadb
pip install faiss-cpu
pip install streamlit
pip install python-multipart
pip install pydantic
pip install pypdf
# Matts Additions
pip install langchain_community

Test OCI package

We will test the OCI package since we have just installed it.

Create a file in your .venv folder called test-oci.py and paste in the following

 import oci

 # Load the default configuration
 config = oci.config.from_file()

 # Initialize the IdentityClient with the configuration
 identity = oci.identity.IdentityClient(config)

 # Fetch and print user details to verify the connection
 user = identity.get_user(config["user"]).data
 print("User details: ", user)

Run the file SHIFT+F10 If it works, you should see some JSON like this

   "inactive_status": null,
   "is_mfa_activated": true,
   "last_successful_login_time": "2024-06-14T08:54:58.148000+00:00",
   "lifecycle_state": "ACTIVE",
   "name": "mmulvaney@leedsunited.com",
   "previous_successful_login_time": null,
   "time_created": "2022-11-01T14:50:28.449000+00:00"
 }

 Process finished with exit code 0

Configure Chatbot Code

Download the repository as a zip file - click this link
Unzip the contents to your .venv folder
Right-click the project > Reload from Disk
It should look like this
Download some PDFs. I'm using this PDF (Yes, I do own all these consoles)
Add them to new pdf-docs folder, which should be a sub-folder of module4
Open demo-chroma-create.py
Change the URL (Chicago, Frankfurt or other) and the Compartment ID

If you don't know how to obtain the Compartment, it should be visible in Playground > View Code > Python
Run the file and check you have a non-zero number of documents loaded into chromadb

Note: You get this warning as specifically, as of Chroma version 0.4.x, documents are automatically persisted, and the manual persist() method is deprecated. Therefore you can remove the persist() code or ignore this.

Run the chroma server

chroma run --path .venv\ou-generativeai-pro-main\demos\module4\chromadb

Open demo-ou-chatbot-chroma-final.py in PyCharm
Change the two occurrences of URL (Chicago, Frankfurt or other) and the Compartment ID
Open another Terminal (use the + button) - i.e keep chroma server running

Start the chatbot and hit Enter when it asks for your email

streamlit run .venv\ou-generativeai-pro-main\demos\module4\demo-ou-chatbot-chroma-final.py

This will start your Chatbot
Ask it...
```
What consoles does Matt Mulvaney own?
```
ENJOY!
If you see this error... then Text Generation is not available in your Model. Therefore read the next section on Streamlit Chatting
Note: That you can also run examples from the Playground. Here's an example of me asking what the Nintendo Wii is:

Streamlit Chatting

Note: Only do this if you hit the error above and want to use to use Streamlit with the Chatting Service on Frankfurt or Chicago

Note: this code sample does not use memory for conversation history or chroma for RAG. You are very welcome to provide improvements to this code and I will update this blog.

Create a file in your .venv folder called test-oci.py and paste in the following

You have to change the:

Model ID: Click View Model Details and then copy the OCID
Compartment ID: It should be visible in Playground > View Code > Python

import streamlit as st
import oci


# Step 1: Initialize OCI Generative AI Inference Client
def create_chain():
    config = oci.config.from_file('~/.oci/config', 'DEFAULT')
    endpoint = "https://inference.generativeai.eu-frankfurt-1.oci.oraclecloud.com"
    generative_ai_inference_client = oci.generative_ai_inference.GenerativeAiInferenceClient(
        config=config,
        service_endpoint=endpoint,
        retry_strategy=oci.retry.NoneRetryStrategy(),
        timeout=(10, 240)
    )

    def invoke_chat(user_message):
        chat_request = oci.generative_ai_inference.models.CohereChatRequest()
        chat_request.message = user_message
        chat_request.max_tokens = 600
        chat_request.temperature = 1
        chat_request.frequency_penalty = 0
        chat_request.top_p = 0.75
        chat_request.top_k = 0

        chat_detail = oci.generative_ai_inference.models.ChatDetails()
        chat_detail.serving_mode = oci.generative_ai_inference.models.OnDemandServingMode(
            model_id="[MODEL ID]"
        )
        chat_detail.chat_request = chat_request
        chat_detail.compartment_id = "[COMPARTMENT]"

        chat_response = generative_ai_inference_client.chat(chat_detail)
        return chat_response  # Return the whole chat_response object

    return invoke_chat


# Step 2: Define Streamlit UI
if __name__ == "__main__":
    chain = create_chain()

    st.subheader("Pretius: Chatbot powered by OCI Generative AI Service")
    user_input = st.text_input("Ask me a question")

    if user_input:
        bot_response = chain(user_input)
        if bot_response.status == 200:
            # Ensure bot_response is correctly accessed based on actual structure
            chat_response = bot_response.data.chat_response  # Assuming chat_response is within data attribute
            if chat_response:
                st.write("Question: ", user_input)
                st.write("Answer: ", chat_response.text)  # Adjust based on actual structure
            else:
                st.write("Unexpected response format from OCI Generative AI Service.")
        else:
            st.write("Error communicating with OCI Generative AI Service.")

Start the Chatbot

streamlit run .venv\ou-generativeai-pro-main\demos\module4\demo-pretius-working.py

Enjoy your Chatbot

Deploy the Chatbot

To Deploy the Chatbot, read the OU ChatBot Setup-V1.pdf document found in the module4 folder.

ENJOY

What's the picture? Swans at Golden Acre Park, Leeds. Visit Yorkshire!

Build an OCI Generative AI Chatbot with PyCharm & RAG