Logan:

Enter Logan

In my previous post Ansible meets CML, I presented a rudimentary log analysis tool. Since then its features have evolved significantly, with the addition of retrieval augmented generation (RAG), a better GUI, and multi-format support. After adding these new features, it is now possible to process logs, notes and documents locally and securely. I started calling this project “Logan” because originally it was born from a Log Analyzer. Logan today is a RAG engine that can analyze multiple types of documents.

Logan providing analysis of a log/configuration file for a Ubuntu workstation

Only recently has local LLM technology become economically practical to run on workstation-grade hardware, thanks to advancements in GPU performance and general compute capabilities. This is in addition to advancements in the sophistication of local models. We can now achieve extremely impressive results on consumer hardware — locally. What can be done today on a workstation or server, would have been considered impossible a few years ago.

Why Build it?

I wanted to create a tool that I would find useful, and at the same time, learn about how RAG works. While working on this project, I was able to discover more about RAG, LLMs and Agentic Frameworks.

Having a local tool that can retrieve information from multiple documents fills a niche that can save time and reduce errors. Just as importantly, building my own solution gave me a hands-on understanding of how the underlying technologies function and interact. The process was as valuable as the product, deepening my knowledge while creating something genuinely useful.

Logan pulling accurate information from a collection of 14 “technical notes”

Design Choices

As I have stated in previous posts, I am not a programmer by trade, and putting this project together was an interesting challenge. Some of the design choices I made along the way were inspired by similar projects, educational resources, and public online tools.

Key features:

• Support for multiple data types (logs, PDFs, books, and notes)

• User selectable prompt behavior types, for example optimizing for Accuracy, Concise responses, etc

• Interactive GUI for querying and information retrieval

• Efficient local-first retrieval engine optimized for various document types

• Adjust Temperature and Template Type to allow for inference or direct reference

• Memory manager that allows the user to search for information directly from ingested documents or from previously saved conversations

• Free-Chat Mode: Chat without documents — acts as a local AI assistant, can be used for testing different SLM/LLM models before enabling RAG

• Clear Context & Memory Control**:** Restart conversations with a click

• Support of multiple embedding models

• Ability to choose LLM model and settings

• Designed for expansions and plugins in future versions

• Scalable, user selectable models (pick the best model for your use case and hardware)

---

Six Layer Architecture

1. Data Ingestion Layer: Handles parsing and processing of various data sources (logs, PDFs via PyMuPDF, texts files, and notes)

2. Storage Layer: Manages vector embeddings (FAISS) and raw documents (SQLite) for both short-term and long-term memory

3. Retrieval Layer: Responsible for finding relevant information based on queries, optimized for specific document types

4. Generation Layer: Combines retrieved context with LLM capabilities for accurate responses

5. Interface Layer: Provides desktop GUI (Tkinter) for user interaction

6. Orchestration Layer: Coordinates the flow between components

Under the hood

• Core Components:

  • main.py – Startup and orchestration

  • RAGSystemGUI – GUI frontend with mode switching (Logs, Books, Notes)

  • ChainRouter – Routes queries to the right pipeline

  • RAGPipeline – Handles retrieval, memory, and generation

  • MemoryManager – Tracks conversation state and context

  • SearchEngine – Fast vector similarity search (FAISS)

  • Generator – LLM wrapper and prompt engine (Plugs into ollama)

  • IngestionManager – Parses, chunks, and embeds source files

Storage:

• SQLite for metadata and conversations

• FAISS for fast vector search

Generation:

• Implements Ollama integration for LLM tasks

  More detailed information on project architecture is explored in the readme.md file.

Querying synthetic medical records, temperature set for mild inference ~.28

Use Cases in the Wild

• Network Admin: "What ports are available on Switch90 and what is their throughput?"

• Security Engineer: “Are there any blatant security flaws with my current configuration?”

• Reader/Researcher: “What does the literature say about X?”

• Engineer/Professional: Search your logs, notes, or docs

• Small Business: Keep your data local and still get accurate answers

• Medical: Search and analyze HIPAA sensitive medical documents locally

• Financial: “Do the documents suggest a correlation between X and Y?”

Querying a PDF textbook and retrieving accurate information, with mild inference ~.15

What’s Next for Logan

• Smart reloading and memory compression techniques to optimize context window usage

• Additional prompt templates

• Multi-modal support for image ingestion and retrieval

• Function calling support for multi-agent integration

• Web-based interface option for cloud deployments

• Improved document citation

Looking Ahead

Creating Logan was a great experience for me to learn more about AI fundamentals like LLMs and RAG. I was also able to create a viable tool with real-world application. I look forward to improving the current design and exploring modifications for specific niche use cases.

Access the Git Repository:

https://github.com/fullMetalLarry/LOGAN

Contact Me about this project:

fullMetalLarry@proton.me