The Tools and Plugins I Use to Build Local AI Projects (Without Spending a Fortune)

Hi! I’m a full-stack software engineer building AI tools on the side using just my gaming PC, a 3080, and an obsession with doing things the hard way.

Most of my projects are powered by open models and local-first frameworks—no fancy clusters or cloud credits needed. I do this not just to save money, but because I care about cost transparency, full control, and keeping things private. I’m a sucker for privacy, and I don’t like paying premium prices just to experiment.

If you’re a broke indie dev, a hacker with a GPU, or just someone curious about building smarter tools on a budget—this stack might be all you need.

---

1. Running Local LLMs

Why I Love Ollama—and Where It Struggles

Ollama is the core of my stack. It’s a CLI tool that lets you download and run open LLMs like LLaMA 3, Mistral, and Gemma locally on your machine with minimal setup.

Why I love it:

• Free and fast to iterate

• No API keys, no rate limits, no token anxiety

• Great for testing prompt structures, chaining agents, or hacking late at night

But let’s be honest—there are real bottlenecks:

• RAM + VRAM limits: Even with a 3080, I’m mostly limited to 7B and 13B models unless I quantize them

• Slow response times: Local models are slower than OpenAI’s cloud, especially with large context or multiturn reasoning

• No GPT-4 yet: The best open models still fall short of GPT-4 on complex reasoning or synthesis tasks

Still, Ollama gives me enough capability to build and validate real tools—and honestly, that’s 90% of the game.

---

2. My Dev Stack

How I Build Agents Without Cloud Dependence

I try to keep things minimal and modular. Here’s what I use to prototype:

• Chainlit — A clean chat UI for debugging and building agent frontends

• Semantic Kernel — Handles memory, tool calls, and orchestration in my hackathon project

My typical flow:
Prototype with Ollama + Chainlit → add orchestration with SK → layer on tools and memory

---

3. The Plugins That Supercharge My Local Agents

These tools help my agents go beyond “just generating text”:

• Brave Search API — Used in my Researcher agent to get real-time info. Clean results and generous free tier

• MCP-style plugins — File readers, terminal commands, and API triggers that let agents actually do things

• Discord API — I’ve built bots like a reverse dictionary game and trivia engine. Discord makes AI tools feel more social and interactive

---

4. The Tradeoffs

Why Local Is Still Worth It

Local builds aren’t perfect, but they’re yours. That counts for a lot.

Why I love it:

• No OpenAI overage stress

• No waiting on rate limits

• Full control over memory, tools, and privacy

Why it’s still a grind:

• Slower prototyping for complex flows

• More dev ops overhead (Ollama can still hang or crash)

• Model selection is limited unless you’ve got 24+GB of VRAM

Even with the rough edges, I’ll take this setup over cloud dependence any day.

---

5. What I’ve Actually Shipped

This isn’t just theory—I’ve built and open-sourced:

• Researcher + Synthesizer POC
  Two agents using Brave Search + Ollama to research and summarize info with simple memory and plugins.

• Discord GPT guessing game (in progress)
  A fun bot I’m building that describes a word—you guess it. The core logic is working, and it’s already fun in testing.
  If you want to try it out and help me polish it, just DM me and I can add you to the testing Discord.

• Secret hackathon project
  Can’t say much yet, but it involves multi-agent orchestration, memory shaping, local plugins, and a clean UI. Full breakdown coming soon.

---

TL;DR

• Local tools are good enough to build real things

• Ollama + Brave + Chainlit + SK = scrappy but powerful stack

• A 3080 is enough to build, prototype, and ship—if you understand the limits

• You don’t have to chase every trend. Just build what feels right for you

---

Let’s Connect

If you’re building cool stuff locally—or want to start—I’d love to hear about it.

Code samples and tools:
https://github.com/chin3/

Follow my devlog:
I post about solo agent systems, local-first builds, and AI experiments on Substack / Hashnode / Youtube(WIP)

Reach out to me on twitter https://x.com/crimsun_chin, I’ll add my discord here too later

If this helped you, share it with a dev friend who needs a nudge to finally start their idea.