Feed
Discussions

llamacpp

llamacpp

#llamacpp

3 followers·8 articles

llamacpp

#llamacpp·3 followers·8 articles

llamacpp

Changelog

New steps component and improved accessibility on Hashnode's blog and docs product.

Nov 05, 2024·

new

Trending Articles

The Stablecoin Surge: How CryptoXpress Is Leading the Charge Toward Financial Liberation

Naomi Antia Benson·55 reads

CryptoXpress: Because Managing Crypto Shouldn't Feel Like Filing Taxes

Prochino·72 reads

XPRESS Delisting on Gate.io: What’s Happening and What’s Next.

Patience Uyire·32 reads

Top commenters this week

Writing Challenges

#2Articles1Week Challenge

Become better at technical writing; accept Hashnode's writing challenge for four weeks.

#2Articles1Week Challenge

#WomenWhoTech

Share your story, achievements, or experiences as a woman, non-binary folk in tech or as a #WomenWhoTech ally!

#WomenWhoTech

Self Starter

Publish your first article on Hashnode and become a self starter!

Self Starter

Serial Blogger

Publish an article every day for 7 days and earn a cool serial blogger badge!

Serial Blogger

Talk of the town

Write a story that drives amazing engagement on Hashnode and become the talk of the town!

Talk of the town

Word Warrior

Write an in-depth article on your Hashnode blog that's more than 2000 words and become a word warrior!

Word Warrior

Sumayya Irfan

Rishav

Marketer lab

Astha Sharma

Deepak Kumar

Pranjali Sachan

Pranjali Sachan

pranjalisachan.hashnode.dev·Mar 23, 2025

Mar 23, 2025

Deploying Gemma 1B GGUF model with llama.cpp on Ubuntu

For this guide we should be using an Ubuntu server with atleast 2 GiB of GPU VRAM, 2 VCPU Cores, 8 GiB RAM, and 50 GiB of storage. Let’s start by updating and installing necessary libraries. sudo apt update && sudo apt upgrade -y sudo apt install -y ...

Deploying Gemma 1B GGUF model with llama.cpp on Ubuntu

Discuss·37 reads

Deploy Quantized Models with llama.cpp on Ubuntu

Ankit Babber

blogababber.hashnode.dev·Feb 11, 2025

Feb 11, 2025

Run a LLM Locally on an Intel Mac with an eGPU

Originally published on https://www.ankitbabber.com I have a Mac with Intel silicon. I also have an eGPU with an AMD 6900XT (...allright!). BUT I COULDN'T HARNESS THAT POWER AND RUN A LLM LOCALLY WITH OLLAMA!!! If you have a Mac with Intel silicon, ...

Discuss·24 reads

Robert Collins

selfenrichment.hashnode.dev·Oct 28, 2024

Oct 28, 2024

On KL-Divergence and Context Size Optimization in GGUF Quantization

note: This paper was written with AI, and the exploration it describes was done collaboratively with AI. The "we" described here is us; me and a few models. With llama.cpp model quantization, properly adjusting models to keep their performance after ...

Discuss·150 reads

Amos Gyamfi

amosgyamfi.hashnode.dev·Aug 29, 2024

Aug 29, 2024

The 6 Best LLM Tools To Run Models Locally

Running large language models (LLMs) like ChatGPT and Claude usually involves sending data to servers managed by OpenAI and other AI model providers. While these services are secure, some businesses prefer to keep their data entirely offline for grea...

Discuss·8 reads

Shivang Agarwal

Shivang Agarwal

shivangagarwal.hashnode.dev·Feb 29, 2024

Feb 29, 2024

How to Run LLMs Locally

Introduction As the Open Source LLMs evolve, the power of Generative AI is within reach of all. Against the common belief that you need mighty machines or graphic cards to use these models, many techniques have evolved that allow you to run these mod...

How to Run LLMs Locally

Discuss·69 reads

Akriti Upadhyay

Akriti Upadhyay

akritiu.hashnode.dev·Feb 02, 2024

Feb 02, 2024

Integrating LlamaIndex and Qdrant Similarity Search for Patient Record Retrieval

Introduction The medical field is currently experiencing a remarkable surge in data, a result of the progress in medical technologies, digital health records (EHR), and wearable health devices. The ability to effectively manage and analyze this intri...

Integrating LlamaIndex and Qdrant Similarity Search for Patient Record Retrieval

Discuss·4 reads

Lars Grammel

p42.hashnode.dev·Jan 13, 2024

Jan 13, 2024

Create Your Own Local Chatbot with Next.js, Llama.cpp, and ModelFusion

In this blog post, we'll build a Next.js chatbot that runs on your computer. We'll use Llama.cpp to serve the OpenHermes 2.5 Mistral LLM (large language model) locally, the Vercel AI SDK to handle stream forwarding and rendering, and ModelFusion to i...

Create Your Own Local Chatbot with Next.js, Llama.cpp, and ModelFusion

Discuss·6 likes·1043 reads

Kaushal Powar

writtenbykaushal.hashnode.dev·Jan 04, 2024

Jan 04, 2024

How to convert HF (safetensors) 🤗 model to gguf

You want to convert Huggingface model to gguf format?I was struggling to tackle the same problem a few days ago. I finetuned a Llama 7B model and the model was saved in safetensor format. I wanted to use gguf model so I searched a lot and found a sol...

How to convert HF (safetensors) 🤗 model to gguf

Discuss·1 like·7826 reads