Analyzing logs to diagnose production issues is something every developer or SRE team deals with regularly. Most organizations have a centralized logging system with visualization tools like Kibana, Grafana, or Datadog. However, even with those in place, it can still be tough to triage complex application problems—especially when the issue spans across multiple services or involves subtle failures that aren't easily surfaced in dashboards.

Often, we fall back on traditional UNIX commands (like grep, awk, and tail) for quick log investigation. But I recently wondered: What if I could use a local LLM (Large Language Model) to help me analyze logs? Could I feed the logs into an AI system and simply ask questions to pinpoint the root cause faster?

In this blog post, I'll walk through a proof of concept I built using an LLM running locally. I'll show you how I set up the environment, parsed and embedded logs, and used the model to answer questions interactively.

Local set up

I wanted everything to run locally—no API limits, no internet latency, and no need for the latest bleeding-edge model. A moderately capable model would be enough for my experiment.

Step to configure your local environment ( I am using mac )

Step 1: Install Ollama (on macOS)

brew install ollama
brew services start ollama

Step 2: Pull and Run the Model

We'll use llama3 for this example:

ollama pull llama3
ollama run llama3

Ollama exposes a REST API at http://localhost:11434. You can test it using:

curl http://localhost:11434/api/generate \
  -d '{ "model": "llama3", "prompt": "Explain Kubernetes in simple terms", "stream": false }'

Step 3: Install Dependencies

We’ll use LangChain, FAISS (for vector storage), and some related Python libraries.

ollama pull nomic-embed-text

pip install -U langchain langchain-community langchain-ollama
pip install faiss-cpu ollama openai tiktoken

Once you have LLM setup done locally we move to our code where we will feed the logs to vector store and then LLM and then start an interactive prompt and evaluate how it responds.

Architecture

Here’s what the end-to-end flow looks like:

cssCopyEdit[ .log files ]
       ↓
[TextLoader]
       ↓
[Text Splitter]
       ↓
[Embedding Model (Ollama)]
       ↓
[Vector Store (FAISS)]
       ↓
[Retriever]
       ↓
[You ask a question]
       ↓
[Relevant chunks retrieved]
       ↓
[Local LLM (Ollama) answers]

Here are the list of common libraries we will be using

Vector database FAISS
Framework langchain
We need nomic-embed-text which langchain will use from ollama. We need to pull it explictly
- ollama pull nomic-embed-text
We need to install langchain
- pip install -U langchain langchain-community langchain-ollama
We also need to pull other dependencies
- pip install faiss-cpu ollama openai tiktoken

Now its time to write the actual code.

Implementation

Goals:

Read .log files from a directory.
Tokenize and parse each log line.
Embed the chunks using Ollama.
Store the vectors in FAISS.
Accept user queries in CLI and return relevant answers.

Lets look at the code first.

import os
import re
from langchain.document_loaders import TextLoader
from langchain.embeddings import OllamaEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.llms import Ollama
from langchain.schema import Document

LOG_DIR = "/path/to/logs"  # <-- Adjust path as needed

# Define your regex pattern (escaped)
log_line_pattern = re.compile(
    r"\[(.*?)\] (\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2}\.\d{3}) \[(.*?)\] ([^ ]*) +([^ ]*) ([^ ]*) (.*?) (.*)"
)

# STEP 1: Parse each line of the log file
def process_log_line(line):
    match = log_line_pattern.match(line)
    if match:
        groups = match.groups()
        if len(groups) == 9:
            level, date, time, thread, session_id, request_id, clazz, dash, message = groups
            return f"""Log Level: {level}
Date: {date}
Time: {time}
Thread: {thread}
Session ID: {session_id}
Request ID: {request_id}
Class: {clazz}
Message: {message}
"""
    # fallback if line doesn't match the pattern
    return f"Unstructured Log: {line.strip()}"

# STEP 2: Load and chunk logs by line
def load_all_log_files(log_dir):
    all_docs = []
    for root, _, files in os.walk(log_dir):
       for file in files:
            if file.endswith(".log") or ".log." in file:
                path = os.path.join(root, file)
                try:
                    if os.path.getsize(path) == 0:
                        print(f"Skipping empty file: {path}")
                        continue
                    with open(path, "r", encoding="utf-8") as f:
                        for line in f:
                            text = process_log_line(line)
                            doc = Document(page_content=text, metadata={"source": file})
                            all_docs.append(doc)
                    print(f"Loaded {len(all_docs)} lines from: {path}")
                except Exception as e:
                    print(f"Error loading {path}: {e}")
    return all_docs

# STEP 3: Create vector store
def create_vector_store(docs):
    embeddings = OllamaEmbeddings(model="nomic-embed-text")  
    return FAISS.from_documents(docs, embeddings)

# STEP 4: Build LLM QA chain
def build_qa_chain(vector_store):
    llm = Ollama(model="llama3")
    return RetrievalQA.from_chain_type(llm=llm, retriever=vector_store.as_retriever())

# STEP 5: CLI interaction
def main():
    print("Reading logs from:", LOG_DIR)
    docs = load_all_log_files(LOG_DIR)
    if not docs:
        print("No log content loaded.")
        return

    vectordb = create_vector_store(docs)
    qa_chain = build_qa_chain(vectordb)

    print("\n Ready to answer questions from logs. Type 'exit' to quit.\n")
    while True:
        query = input(" Your question: ")
        if query.lower() in ['exit', 'quit']:
            break
        response = qa_chain.run(query)
        print("Answer:", response, "\n")

if __name__ == "__main__":
    main()

Save this as log_analyser.py, then run it:

python log_analyser.py

Now try to ask questions regarding the logs and see the output.

Observations

Initially, I tokenized and fed logs line-by-line, but the LLM gave unsatisfactory responses.
Adding structure to the logs (e.g., parsing key fields) helped slightly, but the results were still not insightful for complex, contextual questions.
It struggled with queries like:
- “How long did operation X take?”
- “Do you see any anomalies?”
It seems that for deeper insights, the model needs more metadata and perhaps domain context.

Conclusion

While this POC could answer simple questions and summarize log entries, it struggled with deeper contextual analysis. To get more meaningful results, I may need to:

Enrich the logs with more structured context.
Correlate events across services.
Provide higher-level summary data or KPIs.
Explore chaining techniques or prompt engineering.

Still, I found the approach promising. If you try this experiment or tweak the setup, I’d love to hear about your results!

Stay tuned—I'll continue refining this and share my progress in the next blog post.

My experiment with analysing logs using LLM