Ollama¶

Ollama makes it easy to get up and running with large language models locally.

Setup¶

SSH into the machine running HomelabOS and install a model like so:

docker exec -it ollama_ollama_1 bash
ollama run llama2

You can then make direct requests within the container using standard ollama interface, or hit the external CURL endpoint, like:

curl https://ollama.nerdcave.tv/api/chat -d '{
  "model": "llama2",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

This endpoint is publicly accessible, with no rate limiting. It could lead to pain.

To enable https_only or auth set the service config to True settings/config.yml

ollama:
  https_only: True
  auth: True

Last update: September 26, 2026