A while back, I realized I was growing too dependent on cloud-based AI tools. They were fantastic for writing shell scripts, refining markdown docs, and explaining obscure error logs, but the limitations piled up. Usage caps, network lag, recurring charges, and that uneasy feeling that my prompts were being recorded somewhere in a server farm.

So I went looking for something different, an AI solution that didn’t need the cloud, didn’t bill me monthly, and respected my privacy. That’s when I discovered Jan, a self-hosted AI interface that runs directly on your laptop or workstation, and Pinggy, a lightweight way to securely share local apps on the internet.

In this post, I’ll walk you through how I got everything up and running from model installation to API access to secure remote sharing using just Jan and Pinggy.

Meet Jan: An Offline AI Chat Assistant

Jan is a desktop AI application that mimics the experience of using ChatGPT but operates completely on your device. It’s available for macOS, Linux, and Windows. With Jan, there’s no dependency on browsers, no online account needed, and no data leakage.

It supports open LLMs that you can download and run locally. You can also connect to commercial models from OpenAI, Anthropic, and others if you need to augment local capabilities.

Why Developers Should Consider Jan

Jan isn’t just for AI hobbyists; it’s a practical tool for professionals who need:

Private computation: Your queries remain local
Zero vendor lock-in: No need to subscribe or depend on third-party platforms
Offline access: Works without an internet connection
Open codebase: You can inspect, modify, or fork as needed
Flexible integration: Comes with an API for plugging into custom workflows

It’s particularly useful for developers and teams working in secure or air-gapped environments.

Step 1: Installing Jan Locally

Start by downloading the latest build from jan.ai. You’ll find standalone executables for all major operating systems. No containerization or virtual environments are required; just launch and go.

Prefer to compile from source? You can do that too:

git clone https://github.com/menloresearch/jan
cd jan
make dev

Once opened, Jan greets you with a clean UI. But to get started, you’ll need a model.

Step 2: Load a Language Model

Inside Jan, head over to the Hub section. From there, you can browse through several language models in GGUF format.

Choose one based on your system specs:

For machines with limited RAM, try a compact model like qwen2.5
If you’ve got more memory (8GB or more), try phi3.5 or something larger

All downloads are handled in-app. Once a model is available, it will show up in your selection menu.

Pro Tip: Validate your setup with a small model before downloading the more resource-intensive ones.

Step 3: Begin Conversing with Jan

With a model ready, go to the New Chat tab and begin typing. Jan processes your input right there on your system. You can use it to refactor code, explain errors, write technical docs, or even simulate interviews.

Everything runs locally, so you’re never sending data to a remote server unless you explicitly opt in.

Step 4: Connect to Remote AI Providers (Optional)

Jan supports connecting to commercial LLM APIs for tasks that require more advanced reasoning or fluency. You can enable providers like:

OpenAI
Anthropic (Claude)
Google (Gemini)
Groq
Cohere

Here’s how:

Open Settings → Model Providers
Choose your provider
Enter your API key

Now you can switch between local and hosted models as your needs change.

Step 5: Turn On the API Server

Want to use Jan in a backend project or local dashboard? You can enable its built-in REST API, which follows the same schema as the OpenAI API.

To activate the API:

Navigate to Settings → Local API Server
Set an API key (e.g., jan-dev-key)
Click “Start Server”

You’ll get a response like:

JAN API listening at: http://127.0.0.1:1337

Example usage with curl:

curl --location 'http://127.0.0.1:1337/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer jan-dev-key' \
--data '{
  "model": "qwen3:0.6b",
  "messages": [{"role": "user", "content": "What’s a mutex in C++?"}]
}'

Step 6: Make Jan Public with Pinggy

Sometimes you’ll want to share your Jan instance, maybe to test it on a different machine or let a colleague try it. That’s where Pinggy comes in.

With Jan running on your machine, just open your terminal and run:

ssh -p 443 -R0:localhost:1337 -t qr@a.pinggy.io "u:Host:localhost:1337"

Pinggy instantly gives you a live, public URL like:

https://your-pinggy-url.pinggy.link

Open this in any browser, and your assistant is available remotely, served directly from your device.

Securing Your Shared Assistant

Pinggy supports basic authentication. Add a username and password like so:

ssh -p 443 -R0:localhost:1337 -t a.pinggy.io b:myuser:mypass

This ensures your public URL isn’t exposed to just anyone.

For more professional setups, Pinggy Pro offers custom domain support.

When This Setup Shines

Here’s where I’ve personally used Jan + Pinggy effectively:

On-site automation: Automate config file generation and scripting in secure networks
Team demos: Share a temporary AI assistant during client walkthroughs
Remote access: Use the same assistant across multiple machines
Dev environment helper: Query logs, build snippets, or test APIs locally

Whether you’re prototyping, debugging, or exploring, it’s a reliable sidekick.

Conclusion

If you’ve been looking for an AI solution that respects your privacy, doesn’t cost a dime, and gives you full administrative access, Jan delivers.

Pair it with Pinggy, and you can take your assistant on the road or share it with teammates, without setting up a cloud server or exposing your internal network.

The setup is lean, the performance is solid, and the flexibility is unmatched.

Deploying a Local AI Assistant with Jan and Enabling Remote Access via Pinggy