Private LLM (Ollama)

An Ollama Egg runs the Ollama runtime on a host you have enrolled yourself, with a persistent volume for the models you pull. It serves two APIs on port 11434: the native Ollama API under /api, and an OpenAI-compatible API under /v1, so most existing clients work by changing one base URL. It has no public front door: you reach it from your other Eggs over the Coop private network, or from its own shell.

Lay one

1. Enrol the host you want the model to run on, if you have not already. See Bring your own host.
2. In the dashboard choose Lay a new Egg, then the Private LLM type. There is no repo and nothing to build.
3. Set the model storage size. Model files are gigabytes each, so allow roughly 5 GB per small quantised model you intend to keep. Storage can grow later but never shrink.
4. Set the memory and vCPU. Roughly 8 GB of memory suits a 7B model and 10 GB a 13B, plus headroom for the host itself.
5. Lay the Egg. It runs our Ollama image and comes up with no models: the first pull is yours to make.

Pull a model

Open the Egg's Shell tab (or use ruust shell from the CLI) and pull the model you want. It downloads onto the Egg's volume, so it survives restarts, redeploys and resizes. We ship the runtime, never the models: you choose what to pull and accept its licence, which for some open-weight families carries real conditions.

ollama pull llama3.2
ollama list

The private AI stack

The pattern worth building is a web Egg that calls the LLM Egg over the Coop private network, so prompts never leave the host and the model is never exposed to the internet. Put both Eggs in the same Coop, turn on private networking for each, and connect them. See Private networking for the mechanics.

1. Put the app Egg and the Ollama Egg in the same Coop.
2. Turn on private networking for both, then Connect them from either Egg's panel on the Coop board.
3. On the app Egg, add an environment variable that references the Ollama Egg. Referencing it fills in its private base URL for you, so nothing is hardcoded.
4. Deploy. Your app reads the variable like any other config.

// OPENAI_BASE_URL is set on the app Egg, referencing the Ollama Egg.
const res = await fetch(`${process.env.OPENAI_BASE_URL}/chat/completions`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'llama3.2',
    messages: [{ role: 'user', content: 'Summarise this ticket.' }],
  }),
});
const data = await res.json();

The Egg also exposes the native Ollama API, which is the one to use for pulling models, streaming raw completions or reading embeddings:

curl http://your-llm-egg:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

Sizing and limits

• Memory is a hard limit. A model that does not fit will fail to load rather than swap, so size the Egg for the largest model you intend to run.
• Storage is grow-only: it can be increased but never shrunk. The disk is your own host's, so the size you set is a cap we enforce rather than a line on your bill; the Egg itself is the flat self-hosted fee. Pulling several large models fills a volume quickly, and ollama rm frees space inside the Egg.
• One replica. The models live on a host-local volume, so an Ollama Egg never scales horizontally and never moves host on a resize. If its host runs out of room, the resize is refused rather than silently orphaning your models.
• Egress is unmetered, so streaming long completions to your users costs nothing extra. Per-Egg fair-use rate limits still apply.
• No public hostname and no TLS certificate, because there is no public front door. Anything that needs to be reachable from outside should call it through one of your own web Eggs.