Brain runs **four inference models**, each configured independently. That independence
is the point: one can run on your own hardware while another runs on an API,
so the decision is per model rather than one all-or-nothing choice.

## The four inference models

| Inference model | What it does | Required |
| --- | --- | --- |
| **Embeddings** | Turns text into vectors, on every capture and every query | Yes |
| **Summarise** | Categorises and condenses what arrives into cited summaries | Yes |
| **Enrichment** | A stronger model for the enrichment gate | Optional |
| **Reranking** | Reorders search results for relevance | Yes |

Embeddings and summarising are what make the knowledge base work at all, and
reranking is what makes its answers worth trusting — a deployment should have
all three. Enrichment is the one genuinely optional inference model.

Reranking can technically be left unconfigured, in which case search falls back to a
rank-fusion algorithm and still returns results. We do not recommend running
that way: it is the difference between results ordered by relevance and
results merely retrieved. Where hardware is the obstacle, a cloud endpoint
costs a few hundred milliseconds and solves it.

## The split that matters

Latency and quality pull in opposite directions on different halves of the
system, and the halves divide conveniently:

- **Searching needs to feel instant** — and it is the cheap half. Small
  models, runs anywhere.
- **Ingesting needs to be accurate** — a mistake there is permanent and
  accumulates — and it is the expensive half.

So the natural shape is: keep search local and fast, spend on ingest quality.
That is usually also the cheapest split.

## Embeddings run anywhere

A single search query embeds in roughly 30ms with no GPU at all — the model
is small and the time goes on network overhead rather than computation.
Measured 15 Aug 2026 on one machine, changing only the compute backend:

| Hardware | Search query | Bulk ingest |
| --- | --- | --- |
| CPU only (6 threads) | 34ms | 5.5 texts/sec |
| Integrated GPU | 31ms | 11.7 texts/sec |
| Discrete GPU | 24ms | 94.5 texts/sec |

**Searching is equally fast on all three.** A GPU only changes bulk ingest,
and even then it is a one-time cost: importing about 7,000 documents takes
roughly 21 minutes on a bare CPU, 10 minutes on an integrated GPU, and 75
seconds on a discrete one. Inconvenient, not prohibitive.

## Reranking is the exception

A reranker does far more work than an embedder — a full pass per
query-document pair, rather than one pass per document — and it shows:

| Hardware | 30-candidate result set |
| --- | --- |
| CPU only | 11.7 seconds |
| Integrated GPU | 7.7 seconds |
| Discrete GPU | 134ms |
| Cloud API | 417–482ms |

The first two are unusable for interactive search. So without a discrete GPU,
reranking belongs in the cloud — 400ms over a network beats 11.7
seconds locally, and it is the one model where that trade is obvious.

## Where each model runs

Running every model on your own hardware — nothing leaving the machine —
needs roughly 16 GB or more of VRAM to hold the ingest models concurrently. That is
how Levirge runs it.

**Any of them can point at a cloud endpoint instead**, including embeddings and
reranking. One API key covers all of them together, so a
mixed deployment is a matter of configuration rather than a different
product.

Mixing is usually the right answer. The common shape is embeddings local —
they run fine on a CPU — with reranking and the ingest models in the cloud,
which is what lets Brain run on a small server rather than a workstation.

## Next

[Requirements](/brain/docs/on-premise/requirements) covers the hardware and
the services around the models. If you know which models you want where,
register below and say so — that is the fastest route to a concrete answer.