Logan:


Enter Logan
In my previous post Ansible meets CML, I presented a rudimentary log analysis tool. Since then its features have evolved significantly, with the addition of retrieval augmented generation (RAG), a better GUI, and multi-format support. After adding these new features, it is now possible to process logs, notes and documents locally and securely. I started calling this project “Logan” because originally it was born from a Log Analyzer. Logan today is a RAG engine that can analyze multiple types of documents.
Logan providing analysis of a log/configuration file for a Ubuntu workstation
Only recently has local LLM technology become economically practical to run on workstation-grade hardware, thanks to advancements in GPU performance and general compute capabilities. This is in addition to advancements in the sophistication of local models. We can now achieve extremely impressive results on consumer hardware — locally. What can be done today on a workstation or server, would have been considered impossible a few years ago.
Why Build it?
I wanted to create a tool that I would find useful, and at the same time, learn about how RAG works. While working on this project, I was able to discover more about RAG, LLMs and Agentic Frameworks.
Having a local tool that can retrieve information from multiple documents fills a niche that can save time and reduce errors. Just as importantly, building my own solution gave me a hands-on understanding of how the underlying technologies function and interact. The process was as valuable as the product, deepening my knowledge while creating something genuinely useful.
Logan pulling accurate information from a collection of 14 “technical notes”
Design Choices
As I have stated in previous posts, I am not a programmer by trade, and putting this project together was an interesting challenge. Some of the design choices I made along the way were inspired by similar projects, educational resources, and public online tools.
Key features:
Support for multiple data types (logs, PDFs, books, and notes)
User selectable prompt behavior types, for example optimizing for Accuracy, Concise responses, etc
Interactive GUI for querying and information retrieval
Efficient local-first retrieval engine optimized for various document types
Adjust Temperature and Template Type to allow for inference or direct reference
Memory manager that allows the user to search for information directly from ingested documents or from previously saved conversations
Free-Chat Mode: Chat without documents — acts as a local AI assistant, can be used for testing different SLM/LLM models before enabling RAG
Clear Context & Memory Control**:** Restart conversations with a click
Support of multiple embedding models
Ability to choose LLM model and settings
Designed for expansions and plugins in future versions
Scalable, user selectable models (pick the best model for your use case and hardware)
Six Layer Architecture
Data Ingestion Layer: Handles parsing and processing of various data sources (logs, PDFs via PyMuPDF, texts files, and notes)
Storage Layer: Manages vector embeddings (FAISS) and raw documents (SQLite) for both short-term and long-term memory
Retrieval Layer: Responsible for finding relevant information based on queries, optimized for specific document types
Generation Layer: Combines retrieved context with LLM capabilities for accurate responses
Interface Layer: Provides desktop GUI (Tkinter) for user interaction
Orchestration Layer: Coordinates the flow between components
Under the hood
Core Components:
main.py
– Startup and orchestrationRAGSystemGUI
– GUI frontend with mode switching (Logs, Books, Notes)ChainRouter
– Routes queries to the right pipelineRAGPipeline
– Handles retrieval, memory, and generationMemoryManager
– Tracks conversation state and contextSearchEngine
– Fast vector similarity search (FAISS)Generator
– LLM wrapper and prompt engine (Plugs into ollama)IngestionManager
– Parses, chunks, and embeds source files
Storage:
SQLite for metadata and conversations
FAISS for fast vector search
Generation:
Implements Ollama integration for LLM tasks
More detailed information on project architecture is explored in the readme.md file.
Querying synthetic medical records, temperature set for mild inference ~.28
Use Cases in the Wild
Network Admin: "What ports are available on Switch90 and what is their throughput?"
Security Engineer: “Are there any blatant security flaws with my current configuration?”
Reader/Researcher: “What does the literature say about X?”
Engineer/Professional: Search your logs, notes, or docs
Small Business: Keep your data local and still get accurate answers
Medical: Search and analyze HIPAA sensitive medical documents locally
Financial: “Do the documents suggest a correlation between X and Y?”
Querying a PDF textbook and retrieving accurate information, with mild inference ~.15
What’s Next for Logan
Smart reloading and memory compression techniques to optimize context window usage
Additional prompt templates
Multi-modal support for image ingestion and retrieval
Function calling support for multi-agent integration
Web-based interface option for cloud deployments
Improved document citation
Looking Ahead
Creating Logan was a great experience for me to learn more about AI fundamentals like LLMs and RAG. I was also able to create a viable tool with real-world application. I look forward to improving the current design and exploring modifications for specific niche use cases.
Access the Git Repository:
https://github.com/fullMetalLarry/LOGAN
Contact Me about this project:
fullMetalLarry@proton.me
Subscribe to my newsletter
Read articles from Evan O'Brien directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by