Model options

Brain runs four inference models, each configured independently. That independence is the point: one can run on your own hardware while another runs on an API, so the decision is per model rather than one all-or-nothing choice.

The four inference models

Embeddings and summarising are what make the knowledge base work at all, and reranking is what makes its answers worth trusting — a deployment should have all three. Enrichment is the one genuinely optional inference model.

Reranking can technically be left unconfigured, in which case search falls back to a rank-fusion algorithm and still returns results. We do not recommend running that way: it is the difference between results ordered by relevance and results merely retrieved. Where hardware is the obstacle, a cloud endpoint costs a few hundred milliseconds and solves it.

The split that matters

Latency and quality pull in opposite directions on different halves of the system, and the halves divide conveniently:

• Searching needs to feel instant — and it is the cheap half. Small models, runs anywhere.
• Ingesting needs to be accurate — a mistake there is permanent and accumulates — and it is the expensive half.

So the natural shape is: keep search local and fast, spend on ingest quality. That is usually also the cheapest split.

Embeddings run anywhere

A single search query embeds in roughly 30ms with no GPU at all — the model is small and the time goes on network overhead rather than computation. Measured 15 Aug 2026 on one machine, changing only the compute backend:

Searching is equally fast on all three. A GPU only changes bulk ingest, and even then it is a one-time cost: importing about 7,000 documents takes roughly 21 minutes on a bare CPU, 10 minutes on an integrated GPU, and 75 seconds on a discrete one. Inconvenient, not prohibitive.

Reranking is the exception

A reranker does far more work than an embedder — a full pass per query-document pair, rather than one pass per document — and it shows:

The first two are unusable for interactive search. So without a discrete GPU, reranking belongs in the cloud — 400ms over a network beats 11.7 seconds locally, and it is the one model where that trade is obvious.

Where each model runs

Running every model on your own hardware — nothing leaving the machine — needs roughly 16 GB or more of VRAM to hold the ingest models concurrently. That is how Levirge runs it.

Any of them can point at a cloud endpoint instead, including embeddings and reranking. One API key covers all of them together, so a mixed deployment is a matter of configuration rather than a different product.

Mixing is usually the right answer. The common shape is embeddings local — they run fine on a CPU — with reranking and the ingest models in the cloud, which is what lets Brain run on a small server rather than a workstation.

Next

Requirements covers the hardware and the services around the models. If you know which models you want where, register below and say so — that is the fastest route to a concrete answer.