My Own Private AI Agent

If all HAL could do was chat, 2001: A Space Odyssey would’ve been a much less interesting movie. Generative AI (GenAI) is rapidly evolving beyond simple conversations—into AI agents that can take action, automate workflows, and make decisions based on user input and learned behavior. After completing Kaggle’s 5-Day Generative AI Intensive Course with Google, I was inspired by the examples of agents with function-calling capabilities—and decided to recreate one using free and open-source software (FOSS). What surprised me most was how effortlessly the core components came together using open-source tools:

• LangGraph is the core framework powering the agent’s decision-making loop. It provides a flexible way to build stateful, multi-step agents by defining a directed graph where each node represents a function or tool the agent can use. I used the prebuilt graph create_react_agent, which allows the agent to choose actions reactively based on intermediate outputs and observations. This method binds the tools—such as SQL queries or API wrappers—to specific nodes in the graph. The agent determines which tool to call at each step and loops until a final result is returned. It’s elegant, composable, and surprisingly intuitive to work with, making it a great fit for chaining multiple operations in a controlled yet dynamic workflow.

• Ollama is a game-changer when it comes to running large language models locally. It abstracts away the complexity of downloading, serving, and managing models with a simple CLI. With just one command, you can pull and run models on your local machine without writing custom backend code. It provides an OpenAI-compatible API interface, making integration with existing tools seamless. For developers who want to build AI agents without relying on cloud-based APIs or incurring usage fees, Ollama makes self-hosting language models incredibly accessible and developer-friendly.

• Qwen2.5-7B turned out to be the best model for the job. I tried several open-source models of similar size for tool calling, and Qwen2.5-7B stood out by a large margin. Its ability to understand tool descriptions, follow structured prompts, and decide when to call and stop calling tools made it significantly more reliable and efficient in practice. Whether the agent needed to run a SQL query or retrieve structured data, Qwen2.5-7B demonstrated a clear grasp of context and intent, making it an excellent choice for this kind of multi-step orchestration.

Running the agent is straightforward once everything is set up. When provided with clear instructions, it typically responds with accurate answers. That said, results can vary. Since it's powered by a language model—not traditional software logic—it doesn’t “debug” in the conventional sense. Instead, refining its performance involves prompt engineering, model selection, and sometimes fine-tuning. It’s a creative, iterative process more akin to training a clever assistant than programming a deterministic system.

Have fun building your own AI agent! You’ll be fine as long as you don’t name it HAL…

👉 GitHub Repo: https://github.com/jeffreychan1/ai-agent