Storage modes

The three modes

You pick the mode at build time with --store-mode. To switch modes later, run rlat convert <km> --to {bundled|local|remote} — it reshapes the storage mode in place without re-running the encoder. The band, registry, and ANN index are all preserved.

Bundled

rlat build ./my-corpus -o my-corpus.rlat --store-mode bundled

Source files are zstd-compressed individually inside the .rlat. The result is one self-contained file. Drop it on HuggingFace Hub, mail it as an attachment, ship it in CI — there is nothing else to fetch.

When rlat search returns a hit from a bundled model, drift_status is almost always verified. A drifted or missing row means the .rlat was edited after build — investigate.

Local (default)

# build (default --store-mode local)
rlat build ./my-corpus -o my-corpus.rlat

# query — source-root defaults to where you built from
rlat search my-corpus.rlat "..."
rlat search my-corpus.rlat "..." --source-root /different/path

The .rlat records (source_file, char_offset, char_length) triples and re-resolves them on disk at query time. The corpus stays in your filesystem; the .rlat just indexes it.

To see only hits whose source still matches the build:

rlat search my-corpus.rlat "..." --verified-only

To re-ingest drifted files in place, use rlat refresh. To keep the archive live as you edit, use rlat watch — the same incremental pipeline, debounced by filesystem events. rlat watch --once is the synchronous CI / pre-commit shape.

Remote

rlat build ./my-corpus -o my-corpus.rlat \
  --store-mode remote --remote-url-base https://example.com/corpus/v1

The .rlat carries a manifest (manifest.json at the top of the ZIP) mapping each source_file to a {url, sha256} pair recorded at build time. The URL is <remote-url-base>/<source_file> (forward-slash-joined). The first time a query touches a passage, the source file downloads to a per-knowledge-model on-disk cache (default ~/.cache/rlat/remote/<km-sha>/) and its SHA is pin-verified against the manifest. Later reads hit the cache. The cache is keyed on (source_file, expected_sha), so a re-pinned file can never serve stale bytes.

The default download timeout is 30 seconds per file — a stalled upstream cannot hang rlat search indefinitely. An SHA-pin mismatch raises an actionable error pointing at the cache file to delete; if upstream genuinely moved, re-run rlat build --remote-url-base ... to regenerate against the current content.

rlat freshness my-corpus.rlat   # read-only: are upstream SHAs still what we pinned?

rlat freshness walks every manifest entry, downloads the upstream bytes, hashes them, and reports per-entry status (verified / drifted / missing). It is a read-only drift check — it never changes the archive — and suitable for CI gating: it exits non-zero when any drift is detected. When freshness reports drift, run rlat sync — the delta-apply: it buckets passages on a stable passage_id, re-encodes only the updated and added passages, and writes atomically. The manifest, band, and registry stay internally consistent end to end. rlat refresh is the local-mode sibling — the same incremental delta-apply, run against the filesystem instead of the manifest.

Cross-mode rules

• rlat compare always uses the base band, regardless of mode. Two knowledge models built in different modes can be compared as long as their base bands were produced by the same pinned encoder revision.
• Switching modes is rlat convert <km> --to {bundled|local|remote} — it reshapes the storage mode in place, with embeddings, drift hashes, and the ANN index preserved; nothing is re-encoded. See the CLI reference for the per-transition flag requirements.
• Drift is per-passage, not per-file: editing one line in a 5,000-line file marks only the passages whose character range actually changed.

Hosted on a Fabric UDF

To share a .rlat with a team — a single hosted copy, no per-user download — the .rlat lives in OneLake and queries go through a Microsoft Fabric User Data Function:

rlat fabric add team=https://<udf-endpoint-url>
rlat search fabric://team/team-docs "..."

This is not a fourth --store-mode. The .rlat itself stays in any of the three modes above; Fabric hosting layers on top. The UDF reads the .rlat from OneLake, runs retrieval server-side, and returns hits over HTTPS. Authentication is Microsoft Entra (device-code by default, service-principal fallback).

Picking a mode

If you are not sure:

• Building a knowledge model for yourself, against your own working repo — use local. You get drift detection and cheap incremental rebuilds as the corpus evolves.
• Publishing for others to download — use bundled if the corpus is self-contained (consumers get a single file); use remote if the source belongs at canonical URLs (e.g. a tagged GitHub release — the manifest pins SHAs so a moved or rewritten upstream fails loud).
• Shipping in CI, a Docker image, or an air-gapped install — use bundled.
• Sharing with a team where each user works in Claude Code or a local IDE — host it on Fabric. The maintainer publishes one UDF, each user runs rlat fabric add and queries via fabric://.

The technical layout — ZIP internals, NPZ band format, manifest schema — is documented in the internal store specification.

What's next