Running your LLM model locally is easy with the new docker model runner.

Pre-requisites:
Docker Desktop 4.40 and later

Usage: docker model COMMAND

Commands:
list: List models available locally

docker model list

pull: Download a model from Docker Hub: ai/smollm2

dockerm model pull ai/smollm2

rm: Remove a downloaded model

docker model rm ai/smollm2

run: Run a model interactively or with a prompt

docker model run ai/smollm2

status: Check if the model runner is running

docker model status

version: Show the current version

docker model version

---

Start building your Generative AI application powered by the Docker Model Runner.

Clone sample project by docker itself
git clone https://github.com/docker/hello-genai.git

Also, follow the steps given in READEME.md file

Replace your model name with the model given in the .env file; in my case, it would be ai/smollm2

final state of .env file

• SUCCESSFULLY built personal AI chat agent

OUTPUT