Building Intelligent Applications: Weather Agent and Local LLM

In this blog, I’ll share two exciting projects we explored in our programming class: building a weather agent using Python and OpenAI and deploying local language models (LLMs) with Ollama, Docker, and FastAPI. These projects demonstrate how APIs and local models can help create intelligent applications that solve real-world problems. Let’s dive in!

---

Part 1: Building a Weather Agent with Python and OpenAI

Overview

The weather agent was designed to provide real-time weather updates for any city. By leveraging Python, OpenAI's API, and the weather service wttr.in, we created an interactive assistant capable of intelligently responding to user queries.

Code Implementation

Here’s the core functionality of the weather agent:

import json
import requests
from dotenv import load_dotenv
from openai import OpenAI
import os

load_dotenv()
client = OpenAI()

def get_weather(city: str):
    print("Tool Called: get_weather", city)
    url = f"https://wttr.in/{city}?format=%C+%t"
    response = requests.get(url)
    if response.status_code == 200:
        return f"The weather in {city} is {response.text}."
    return "Something went wrong"

available_tools = {
    "get_weather": {
        "fn": get_weather,
        "description": "Takes a city name as input and returns the current weather."
    }
}

system_prompt = """
You are a helpful AI Assistant who resolves user queries step by step using available tools.
Output JSON Format:
{
    "step": "string",
    "content": "string",
    "function": "The name of function if the step is action",
    "input": "The input parameter for the function"
}
"""

messages = [{"role": "system", "content": system_prompt}]

How It Works

• The assistant analyzes user queries and selects the appropriate tool (e.g., get_weather).
• It interacts with APIs to fetch real-time data and formats responses for clarity.

---

Part 2: Running Local LLMs with Ollama, Docker, and FastAPI

Why Run AI Models Locally?

Running AI models locally offers several benefits:

• Privacy: Data stays on your machine.
• Cost Efficiency: No cloud service fees.
• Customization: Tailor models to specific needs.

Step-by-Step Guide

Step 1: Setting Up Ollama

Install Ollama using pip:

pip install ollama

Pull the model:

ollama pull gemma3:1b

Step 2: Integrating with FastAPI

We created a REST API service using FastAPI to interact with the local Ollama model.

FastAPI Code

from fastapi import FastAPI, Body
from ollama import Client

app = FastAPI()
client = Client(host='http://localhost:11434')
client.pull('gemma3:1b')  # Pulling the model

@app.post("/chat")
def chat(message: str = Body(..., description="Chat Message")):
    response = client.chat(model="gemma3:1b", messages=[
        {"role": "user", "content": message}
    ])
    return response['message']['content']

Step 3: Dockerizing Ollama

Use Docker Compose for easy deployment:

services:
  ollama:
    image: ollama/ollama:latest
    ports:
      - '11434:11434'
    volumes:
      - models:/root/.ollama/models

volumes:
  models:

Step 4: Testing the API

Test the FastAPI endpoint using curl commands:

curl --location 'http://127.0.0.1:8000/chat' \
--header 'Content-Type: application/json' \
--data '{"message": "Tell me a joke."}'

---

Benefits of Docker

Docker ensures consistent environments across machines, simplifies scaling, and streamlines deployment.

---

Key Learnings

Weather Agent

• Demonstrated how APIs can be combined to create intelligent tools.
• Highlighted modular design principles in software development.

Ollama Deployment

• Showcased the power of local LLMs for privacy and efficiency.
• Illustrated how Docker simplifies complex setups.

---

Conclusion

These projects demonstrate how AI tools can solve real-world challenges—whether creating interactive assistants or deploying local models. If you’ve tried similar setups or have questions about deploying AI models locally, let’s discuss in the comments below!

---

Related Articles

Here are some additional resources to explore related topics:

• Building a Tokenizer in Python
  Learn how to create your own tokenizer in Python for processing text efficiently.

• Understanding GenAI Prompting Through Cricket
  Explore how Generative AI prompting works through an engaging example based on cricket.

---