Resonance Lattice · Docs

The encoder

rlat ships with one encoder. No options, no presets, no decision tree. This page explains what it is, how it installs, and why the choice is fixed.

01 — The recipe

What it is

An encoder is the model that turns a chunk of text into a vector — a list of numbers that captures the text's meaning. rlat uses exactly one, with one fixed configuration:

BackboneAlibaba-NLP/gte-modernbert-base
PoolingCLS — the vector is read from the model's single summary token
Output dimension768 numbers per vector
Max sequence length8192 tokens — long passages are encoded whole, not split
NormalisationL2 — every vector is scaled to unit length, so similarity is plain cosine
Revisionpinned at install time, and recorded inside every knowledge model

Every knowledge model carries the vectors this encoder produced, in its base band. Cross-model search and rlat compare always operate on these vectors — which is the reason the recipe is locked. Two knowledge models built with the same revision are directly comparable; a per-build encoder choice would break that.

02 — Installation

Installing the encoder

The encoder is not bundled in the pip package — it is downloaded and converted on first use. You install it once per machine.

Automatic

The first time you run rlat build, the encoder installs itself. You do not need to do anything. 

pip install rlat[build]
rlat build ./docs -o docs.rlat
# First run: downloads ~700 MB and converts. Later runs reuse the cache.

Manual

For offline environments, CI runners, or pre-staging a build host:

pip install rlat[build]
rlat install-encoder

This downloads the Hugging Face weights, exports them to ONNX, and — on Intel CPUs that have the openvino package available — also converts them to OpenVINO IR.

Pinning a specific revision

rlat install-encoder --revision 1c7b39da7c5a3f0c92c11d6f5cb6a6e7a3e84c2f

You can pass any Hugging Face ref — a commit hash, a branch name, or a tag. Symbolic refs are resolved to a concrete commit hash before caching, so --revision main and --revision <that-hash> land in the same cache directory.

Reinstalling after a corrupted cache

If a download was interrupted or cache files were modified:

rlat install-encoder --force

This regenerates everything in place.

03 — The cache

Cache location

The encoder is stored under:

$XDG_CACHE_HOME is honoured when it is set. You can safely delete the cache; the next rlat build or explicit rlat install-encoder re-creates it.

What's in the cache

After a successful install, the revision directory contains:

<revision>/
├── revision.txt          # the pinned HF commit hash
├── tokenizer.json        # tokenizer used at query time
├── model.onnx            # ONNX export
├── torch/                # HuggingFace snapshot (used by build paths)
└── openvino/             # OpenVINO IR (only on Intel CPUs)

The directory name is the revision hash. That matters because builds record backbone.revision in metadata.json — a knowledge model built today is byte-comparable to one built tomorrow on a different machine, as long as the revision matches.

04 — Runtime selection

Inference runtime

The runtime is the engine that actually runs the encoder. rlat picks it automatically, based on what is installed and what hardware is present:

Your systemRuntime usedWhy
Intel CPU + openvino installedOpenVINO Runtime1.5–2× faster than ONNX on Intel, via AVX-512 + OpenMP
Other CPU (AMD, ARM, Apple Silicon)ONNX Runtime2–4× faster than PyTorch on CPU
NVIDIA GPU + [gpu] extra installedONNX Runtime (CUDA provider)Auto-preferred for build-time batch encoding; helps the query path on very large batches
rlat buildPyTorchBuild paths need the transformers forward pass; auto-uses CUDA when available

You do not normally choose. The runtime is a function of what is installed and what hardware is available, and no flag exposes it on the search path. If the OpenVINO IR is not in the cache — say you pre-staged on an AMD host and copied the cache to an Intel host — rlat falls back to ONNX automatically. You keep working, just without the Intel speedup.

NVIDIA GPU (CUDA)

If you have NVIDIA hardware:

pip install rlat[build,gpu]
rlat install-encoder

The [gpu] extra installs onnxruntime-gpu, which exposes the CUDA execution provider. rlat's ONNX path discovers the available providers at load time and prefers CUDA when it is present. For rlat build, the PyTorch path also checks for CUDA and lands the model on the GPU when it can.

Query-time GPU

For a single query — one text, ~30 tokens — the cost of copying data to the GPU usually outweighs the compute, so CPU runtimes win. CUDA shines on batch encoding: build paths, anything encoding a corpus or many queries at once. The auto-selector never picks the PyTorch runtime for queries, for two reasons: it tends to be slower than ONNX or OpenVINO on single-query work, and torch lives in the optional [build] extra so it cannot be assumed present.

Watch out

[build] alone is not enough for query-time CUDA. With [build] you get a PyTorch path that auto-uses CUDA for builds. Query-time searches still go through the auto-selected ONNX path, which on a CPU-only onnxruntime install runs on CPU regardless of whether PyTorch sees a GPU. To get query-time CUDA, install [gpu] as well so the ONNX path picks up the CUDA provider.

If you want the PyTorch runtime explicitly, from Python:

from resonance_lattice.field.encoder import Encoder
enc = Encoder(runtime="torch")  # requires the [build] extra; uses CUDA if available

When OpenVINO is unavailable

openvino ships in the [build] extra. If you installed plain rlat with no extras, the OpenVINO path is unavailable and rlat uses ONNX everywhere. That is fine for query-time use — you only need [build] to actually build knowledge models.

05 — The single-recipe decision

Why it's locked

One encoder, one configuration, no knobs. The reason is the base band: cross-knowledge-model search and rlat compare only work if every model's vectors are byte-comparable. A per-build encoder choice — a different backbone, a different pooling, a projection knob — would produce vectors that cannot be compared, and the comparison would silently return meaningless numbers.

Encoder presets, pooling toggles, and projection knobs were measured across BEIR-5 and LongMemEval and dropped. A single locked recipe matches or beats the tuned alternatives while keeping every knowledge model mutually comparable — see the benchmarks page.

06 — Troubleshooting

Troubleshooting

Where to go next