Three retrievers, one answer

May 4, 2026 · updated August 19, 2026 · growing · certain · 8 min

Company search has a reputation for being a plumbing problem. Wire up the connectors, dump everything into an index, put a text box on top. I believed a version of this when I started building OpenBeam, which today has 106 connector integrations in its tree, from Slack and Gmail down to industrial protocols like MQTT and OPC UA. The plumbing was real work. But the part that decides whether the product is any good happens in the roughly two hundred milliseconds after someone types “q3 pricing deck” and hits enter, and none of it is plumbing.

The problem is that “q3 pricing deck” is three different questions wearing one trenchcoat. It is a keyword question: there may be a file literally named that. It is a meaning question: the right document might be titled “Autumn revenue review” and never say the word deck. And it is a rare-token question: “q3” carries more information than its three characters suggest, and a retriever that treats it like any other short token will bury the answer. OpenBeam answers all three questions separately, on purpose, and then holds an election.

Two representations of one query

The oldest answer to search is the inverted index. BM25, the ranking function that still powers most of the world’s text search, is a formula from the nineties that rewards documents for containing your exact words, with damping so that a thousand repetitions of “pricing” don’t count a thousand times. It is fast, unreasonably effective, and completely literal. It cannot see that a “revenue review” is a pricing deck.

Embeddings are the opposite trade. A neural model maps the query and every document into a shared vector space, where closeness means similarity in meaning rather than spelling. Now “Autumn revenue review” can be near “q3 pricing deck”. The cost is the mirror image of BM25’s blindness: in vector space, rare exact tokens lose their sharpness. Product codes, error strings and ticket ids, the things people search for most desperately, dissolve into the soup of meaning.

There is a third representation that splits the difference. A learned-sparse model produces, for each text, a bag of tokens with learned weights: like a keyword index, but where the model has decided which words matter and how much. OpenBeam gets this almost for free. The embedding model it uses, BGE-M3, returns both a dense 1,024-dimension vector and a sparse map of token weights from a single forward pass. One model call, two representations, and the second one costs nothing extra.

Three lanes through one index

At query time, the orchestrator fires three Vespa queries in parallel, one per representation. Each lane retrieves up to three times the requested page size, capped at 300 candidates.

The bm25 lane is classic: match the query words against a fieldset of title and content, rank by bm25(title) * 2 + bm25(content). Titles are trusted twice as much as bodies, which is less a theory of relevance than an honest admission about how people name documents they care about.

The dense lane runs an approximate nearest-neighbor search over the 1,024-d vectors using an HNSW index, with angular distance and Vespa’s closeness as the score. HNSW parameters in the schema are modest, 16 links per node and 200 neighbors explored at insert, which is the region where recall is high and index build stays cheap.

The sparse lane is the sneaky one. Its query contains no text operator at all. It matches on team and filters, then ranks by a dot product between the document’s stored token-weight tensor and the query’s, evaluated at rank time against a plain attribute. There is no inverted index over those learned weights. Strictly speaking it is a ranking lane wearing a retrieval lane’s uniform, and it earns its keep exactly where dense retrieval is weakest: rare tokens that the model has learned to weight heavily.

One more thing rides along in all three lanes. Every document was stamped at ingest with an access-control list: a private Slack message carries its team, channel and member ids; a public doc carries a flag. The caller’s identity is resolved to the same id grammar, and every lane’s query gets the same clause ANDed in, so permission trimming happens inside the engine at match time. There is no moment where an unauthorized document exists in a result list waiting to be filtered out.

Adding the scores would be a category error

Now there are three ranked lists, and the tempting mistake is to add the scores. But a BM25 score, a cosine closeness and a sparse dot product live on different scales with different distributions. Vespa’s own schema shows the workaround this forces: OpenBeam’s single-query hybrid profile squashes BM25 through score / (1 + score) just to make it addable to a closeness value. It works, but every such squashing function is a small lie about geometry, and tuning it means redeploying the schema.

The orchestrator takes a different route: throw the scores away and keep the ranks. Reciprocal Rank Fusion scores each document by summing, across lanes, a weight over k plus the document’s rank in that lane:

const rrfContribution = weight * (1 / (k + rank));

That is the entire kernel, with k at 60 and default weights of 0.4 for bm25, 0.4 for dense and 0.2 for sparse. Rank-based fusion is scale-free, so no lane can dominate just by having enthusiastic score units. A document that no single lane loved but every lane liked can beat one champion’s favorite, which is usually the right call for “q3 pricing deck”. And because fusion happens in application code rather than in a rank profile, the weights are request parameters. Reweighting the election is an API call, not a schema deploy.

The fused results also keep their provenance: each hit carries its per-lane components and ranks in the API response, so when a result looks wrong you can see which lane vouched for it and how hard. Debugging a committee requires knowing how everyone voted.

The funnel spends money where it’s cheap to be wrong

Fusion produces one list of about 300 candidates, and everything after it is a funnel that trades candidates for compute. The top 100 go to a cross-encoder, BGE’s reranker, which reads the query and each document together, title plus the first 2,000 characters, and scores actual relevance rather than proximity in an embedding space. Cross-encoders are what retrieval models pretend to be; they are also far too slow to run against an index, which is why they get 100 documents and a 3-second budget instead of a million and none.

The top 50 then pass through a LightGBM learning-to-rank model with 23 features: the retrieval and rerank scores, recency, document length, engagement counts, title match flags, connector type. It retrains on a schedule, per team, on the last 30 days of search behavior, with a minimum sample floor before a team gets its own model. Finally a personalization layer blends in per-user affinities, connectors you use, authors you interact with, at a fixed 15 percent, and respects an opt-out.

The stages are arranged so failure is boring. The reranker lives in a separate Python engine; if that service is down, the fused RRF order stands. If the LightGBM model file is missing, the rerank order stands. If the embedding service itself is unreachable, the public search path falls back to plain BM25, which is still a real search engine rather than an error page. The funnel is not a chain of dependencies. It is a stack of upgrades, each one optional.

Scar tissue

Reading your own schema honestly is humbling, so here is what the code admits if you look closely. The main document schema defines roughly 21 rank profiles; the live query paths use about seven. The rest, engagement profiles, navigational profiles, an enterprise profile with a carefully tuned second phase, are sediment from experiments, still deployed, never queried. The vector fields tell a migration story in their names: the 1,536-dimension fields of an earlier embedding generation sit next to the current 1,024-dimension fields with a version marker and a scheduled re-embedding workflow that walks old documents forward in batches of ten.

The learning-to-rank stage has the most instructive scar. The model was trained on 23 features, but the orchestrator currently sends real values for only the score features; recency, lengths and engagement go in as zeros. The machinery is sound, the wiring is half-connected, and search quality still improves because the score features carry most of the signal. I am leaving that sentence in because the gap between an architecture diagram and a production system is exactly this kind of detail, and pretending otherwise is how engineering blogs become marketing.

There is also a second, smaller search engine in the codebase: a local one, SQLite FTS5 plus an in-memory vector store, blended with min-max normalized scores rather than RRF, built for on-device RAG. It does not talk to the cloud index at all yet. The interface for syncing context between them exists; the implementation does not. That is not a design, it is a to-do list, and the distinction belongs in writing.

What I actually learned

Search quality is not one clever algorithm. It is an election among cheap, biased voters, followed by increasingly expensive judges, arranged so that any judge can fail quietly. The retrievers are deliberately dumb in three different directions, because their blind spots don’t overlap. The expensive models sit at the narrow end of the funnel where their cost is bounded. And the scores are thrown away at the first opportunity, because ranks are the only thing three different theories of relevance can agree to talk about.

Build the committee before you build the genius. The genius goes at the end, where it’s cheap to be wrong.