Ollama

Ollama runs open-weight language models on hardware you own. With it we can answer “can we use AI?” without adding “by sending your source code to another jurisdiction”.

Why we still choose it

For a European company, the hard question about AI is usually jurisdiction more than capability. A prompt containing an unreleased product plan, a patient record or a client’s source code is a data transfer, whatever the product calls it.

Ollama makes running models locally practical. Models are managed much like packages, and the endpoint is OpenAI-compatible, so existing tooling can be pointed at your own machine by changing the base URL.

How we run it

We declare it on NixOS, on a GPU host sized to the models you need, and expose it only on your WireGuard network so nothing is published. From there it is connected to the places people work: editors, Emacs, Redmine’s REST API, internal search, and retrieval over your own document store.

You pay for electricity instead of tokens, and the compliance discussion is short.

Where it falls short

Local open-weight models are not frontier models. For summarising, classifying, drafting and answering questions about internal documents the gap matters little; for the hardest reasoning work it matters a lot. We help you decide clearly which work may leave the building and which may not.