Top 3 ONNX Models for JavaScript

In the early days of on-device AI, running large language models (LLMs) in JavaScript was only a dream. Developers relied on cloud-based APIs, and any form of local execution was limited to tiny, underpowered models. Fast forward to today, and the landscape has changed dramatically. With the rise of ONNX (Open Neural Network Exchange) and efficient runtimes like Transformers.js, WebGPU, and MLC Web-LLM, running powerful models on-device in JavaScript is no longer just possible, it’s practical.

The Best On-Device ONNX Models for JavaScript

With the release of models like Llama 3.2 1B and 3B, running high-performance LLMs in the browser or on low-powered devices, the question is which is the best? Here are the top ONNX models that can run efficiently in JavaScript:

1. Llama 3.2 (1B & 3B) - Meta's Lightweight Powerhouses

Meta's Llama 3.2 models come in small (1B) and medium (3B) sizes, designed for on-device execution. These models are optimized for:

• Low-latency inference on CPUs and GPUs.

• Multilingual support for English, German, French, Spanish, and more.

• Efficient text generation, making them ideal for chatbots, assistants, and offline summarization.

How to run Llama 3.2 in JavaScript using Transformers.js:

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline("text-generation", "onnx-community/Llama-3.2-1B-Instruct");

const messages = [{ role: "user", content: "Tell me a joke." }];

const output = await generator(messages, { max_new_tokens: 128 });

console.log(output[0].generated_text.at(-1).content);

2. Phi-2 - Microsoft’s Compact LLM

Phi-2 is a compact yet powerful model trained on high-quality datasets. It’s a strong competitor in the small LLM space, providing:

• High accuracy for its size.

• Excellent few-shot learning capabilities.

• Optimized ONNX quantization for better on-device efficiency.

Phi-2 can be run efficiently using WebGPU in MLC Web-LLM:

import { CreateMLCEngine } from "@mlc-ai/web-llm";

const engine = await CreateMLCEngine("Phi-2-q4f32_1-MLC");

const messages = [{ role: "user", content: "Explain the universe as a pirate!" }];

const reply = await engine.chat.completions.create({ messages });

console.log(reply.choices[0].message);

3. Mistral 7B - A Strong Open-Weight Alternative

Mistral 7B provides excellent balance between performance and efficiency. It has:

• Superior text understanding and generation.

• A more permissive open-weight license.

• Efficient execution in ONNX with optimized quantization.

Mistral 7B can be deployed in WebLLM using ONNX:

import * as webllm from "@mlc-ai/web-llm";

const engine = await webllm.CreateMLCEngine("Mistral-7B-q4f32_1-MLC");

const messages = [{ role: "user", content: "What’s the best way to learn JavaScript?" }];

const reply = await engine.chat.completions.create({ messages });

console.log(reply.choices[0].message);

Why On-Device AI Matters

Running AI models on-device has several key benefits:

1. Privacy - No user data leaves the device.

2. Lower Latency - No API calls mean instant responses.

3. Offline Capability - Models run without internet access.

4. Cost Efficiency - No need for expensive cloud compute.

Getting Started with ONNX in JavaScript

If you're looking to integrate on-device AI into your JavaScript projects, here’s what you need:

• Transformers.js for running ONNX models in Node.js or the browser.

• WebGPU support for faster inference with MLC Web-LLM.

• ONNX Runtime Web for cross-platform execution.

The Future of On-Device LLMs in JavaScript

As models continue to get smaller and more efficient, we’re entering an era where advanced AI applications can run directly in the browser or on edge devices. Whether it’s chatbots, real-time translation, or AI-powered assistants, the possibilities are endless.

Want to take your AI projects even further? JigsawStack’s Small Models are built with efficiency in mind, leveraging Node.js on the backend to deliver lightning-fast inference while keeping resource usage low. Explore our APIs today!