How the Catalogue Ranks

Last updated: 2 September 2026

A question reaches the library and comes back as a verdict, hit, guess or nothing, with the books that should open and the agent that should answer. This page is the arithmetic in between. It names every constant, prints what the demo fleet prints today, and keeps the measurement where the tool fails its own test, because a threshold nobody can read is a threshold nobody can argue with.

Two scorers, one fusion

Two scorers read every question, and they read different things. The first reads the labels: the Search for line each book carries under its title, the words a person wrote to say which questions the book answers. The second reads the books themselves, the prose, split into chunks. The catalogue Vesta keeps is the first of these, a keyword index; the second is full text search.

The keyword scorer reads the labels

Each word of the question is weighted by inverse document frequency, the standard arithmetic for how much a word says about which file to open: idf = ln(1 + N / (1 + df)), where N is the number of entries and df is how many of them carry the word. A word on one book in a thousand weighs 6.2; a word on every book weighs almost nothing. One matched key is worth 6 × idf, and the weights fall in order: a hand-written key at 6, the title and the file name at 3 each, the prose of the purpose line at 1. A label outranks a title, and a title outranks a sentence that happened to mention the word.

A multi-word key found whole in the question is the strongest signal a single key can give. It is scored before the individual words, at 10 times the mean idf of its words, which is why the section on writing a label below asks for keys written as the phrases somebody types. Words too common to carry meaning, in both languages a fleet writes, are dropped before any of this, and accents are folded, so producao and produção are one word.

The text scorer reads the books

The second scorer is SQLite FTS5 with BM25 over the chunks of every file, so it reaches words nobody put on a label. kb route --hybrid runs both and fuses the two rankings by Reciprocal Rank Fusion: each list contributes 1 / (60 + rank) to every document it ranks, and the sums are compared. Fusion uses positions and ignores the raw scores on purpose. A BM25 value and a keyword score live in different numeric universes, and normalising them into a weighted sum means inventing a conversion factor and then tuning it per corpus. Ranks need no conversion.

Why they are kept apart

Agreement between two lists is evidence only when the lists are looking at different things, so the independence is a design constraint and not a detail of the implementation. The Search for lines and the map file are excluded from the text index: they are the keyword scorer's corpus, and indexing them twice would make one scorer reading one source twice look like two scorers agreeing. A file contributes to the fusion once, at the rank of its best chunk. Before that rule, scoring every matching chunk turned the ranking into which file has the most matching pieces, and a safety protocol that defined the calorie floor in a single table row lost to a longer file that mentioned it three times in passing.

The index on disk

One dependency: rusqlite with SQLite compiled in, so there is no system package to install and nothing to run. SQLite is there because it brings FTS5 with BM25 built in, holds the chunks and their metadata in one file, and stays readable with any sqlite3 binary. kb index writes one file per base at <base>/.kb/index.db, and there is deliberately no flag to point it anywhere else. The file holds nothing that cannot be recomputed from the markdown: delete it and the next kb index rebuilds it, which is what keeps the books the source of truth and the index a disposable projection of them.

Files are content hashed with SipHash, deliberately not cryptographic, because the question is whether a file changed since last time and the adversary is a text editor. An unchanged file costs nothing on reindex. Chunks split at heading boundaries, then any section over about 1800 characters into windows with 300 characters of overlap, cutting at a blank line when there is one, and the heading path travels with each chunk because it carries context the chunk itself rarely repeats.

Above 10,000 chunks the text scorer prunes. A term found in more than a quarter of the chunks leaves the FTS expression, because BM25's own idf sits near its floor there and the longest doclists in the index would be walked to move nothing; the keyword scorer still sees every word. Measured on 2026-08-26 on the 57,486 chunk LongMemEval-V2 enterprise haystack, identical 19-question sets against the unpruned binary: p95 fell from 1,772 ms to 1,264 ms while the median rose from 306 ms to 371 ms, because the document frequency probes cost more than they save there. Below the threshold nothing changes and the demo eval is bit-identical. That haystack is its own regime: the V2 benchmark that built it names its machine as an 11th Gen i5-1135G7 with 16 GB on Windows 11, and no personal fleet measured so far comes near 10,000 chunks.

Intent chooses the owner, agreement chooses the reading

The two scorers are not two attempts at one job. The owner and the verdict come from the keyword ranking alone; the reading list, the passages a person or a model will read, comes from the fusion. Fusion rewards agreement, which is right when assembling passages, because a book both scorers noticed belongs in front of the reader. It is the wrong rule for choosing one owner: a file each scorer ranks fourth beats a file one scorer ranks first, and that halves precision at the top to buy recall nobody asked for at that step. Both answers come back from one call, Memory::ask, so no caller can pair them differently.

$ kb route "quem decide se um deploy pode ir pra producao" examples/demo
question: quem decide se um deploy pode ir pra producao
indexed:  11 entries across 3 agents, 0 aliases

   1.  33.69  zed/knowledge/deploy-checklist.md
      matched: deploy, producao
   2.   7.49  steve/knowledge/hooks-that-hold.md
      matched: decide
$ kb route "quem decide se um deploy pode ir pra producao" examples/demo --hybrid
question: quem decide se um deploy pode ir pra producao
indexed:  11 entries across 3 agents, 0 aliases

  0.033  zed      knowledge/deploy-checklist.md                keywords #1 + text #2
         What a deploy needs before production: ... A deploy without a rollback plan is a bet, not a release. Canary first ...
  0.033  steve    knowledge/hooks-that-hold.md                 keywords #2 + text #1
         Hooks: the first three seconds decide the rest: **Exists to:** Why openings decide retention, and the three hook shapes that hold  A ...

On the demo's first gold question the catalogue separates the two candidates by four and a half to one, 33.69 against 7.49, on the words deploy and producao. The fusion ties them at 0.033 each: the text scorer ranked the marketing note first, so each file is first on one list and second on the other, and 1/61 + 1/62 is the same sum either way. The owner is zed and the verdict is a hit because the keyword ranking says so. The fused list is what the reader is handed, and here it is right to hand over both.

Folding files into an agent

An agent's evidence has three species, computed from the path and carried nowhere as data: memory is what it knows, the knowledge/ and domain folders; skills is how it works, skills/, protocols/ and templates/; tools is what it has the means to use, the declarations under tools/. The fold takes the best file of each species per agent and sums those at most three numbers. Dense memory contributes exactly one number however many files matched, so fifty-nine notes are worth their best one, and a new agent with no memory at all can win on a skills file that names the subject. The rule replaced a plain sum per agent, under which an agent won by mass, many small scores beating one large one. An agent scoring in two species beats an agent scoring the same in one, which is the shape of being prepared rather than merely informed.

Measured on a fleet you do not have

On the private fleet's answer key, 33 questions of which 24 are answerable, the keyword scorer alone names the right file 11 times and the fusion it feeds names it 8 times. Read that as a direction and not as a score. The key's own header records that the keyword lines were tuned against these same questions on the day they were graded, so the keyword column is flattered, and the key lives in a fleet the repository ignores, so a reader cannot re-run it. What survives the bias is the direction: fed by that same flattered scorer, the fusion still lands behind it when the job is to name one file.

The floor, term by term

The verdict is one comparison: the top file's keyword score against a floor. Both sides are made of the same pieces, so here they are, named as the source names them.

entry

One note the catalogue can find: a markdown file with a Search for line. A fleet's size, for everything below, is its number of entries.

N

The number of entries in the fleet being asked.

key

One term on a note's Search for line.

df, document frequency

How many entries carry a given key. A key on one note has df 1.

idf, inverse document frequency

ln(1 + N / (1 + df)), the weight of a key. Rare keys weigh more; a key on every note weighs almost nothing.

idf_unique(N)

idf with df 1, written out: ln(1 + N/2). The weight of a key exactly one note carries, the heaviest a single key can be at this size. 0.41 at one entry, 1.10 at four, 1.87 at eleven, 4.74 at 226, 6.22 at a thousand.

W_KEYWORD

6.0. One matched key is worth 6 × idf.

keyword score

The sum over every key the question matched, plus smaller terms for title and phrase matches. The number kb route prints beside each file.

SCORE_FLOOR

17.5. Measured on a fleet of 226 entries and moved twice by kb eval.

FLOOR_CALIBRATED_AT

226. The two constants are one fact, that 17.5 was the right floor for a fleet of 226 entries, and they are re-derived together or not at all.

floor_for(N)

0.616 × 6 × idf_unique(N), where 0.616 is 17.5 divided by 6 × idf_unique(226). The same floor, restated as 62 percent of what one unique key scores, and carried to every other size in that unit. A test asserts that floor_for(226) equals SCORE_FLOOR to four decimals.

MIN_ENTRIES_TO_ROUTE

2. Below it the verdict is never hit, whatever the score.

Why it scales

idf grows with N, because rarity needs a corpus to be rare in. A fixed 17.5 therefore meant a different number of keys at every size: close to three unique keys on a base of four entries, one on the base it was measured on, and half a key on a thousand, where a word in fifty files clears it alone. The floor got harder as the base shrank and easier as it grew, in exactly the two directions that hurt. Measured on the demo before the change, a base of two to four entries got not one hit on its own gold questions, every one a guess on the right file, and at the full demo two correct answers were still demoted.

A fleet of one has no ruler

With one entry every key has df 1, so every key weighs the same and idf can tell nothing apart; any shared word clears a floor built from that, not because the evidence is good but because there is nothing to compare rarity against. Below MIN_ENTRIES_TO_ROUTE, two, the verdict is at most guess, served with its warning, and kb boot hands over no identity. Two is the structural minimum, the first size at which a word in both notes weighs less than a word in one, which is the first size at which there is a ruler. A larger minimum would refuse bases that are measured to route correctly, and no measurement supports one.

Measured, before and after

Three instruments, the same binary either side of the change, 2026-09-02. The demo, 11 entries and 13 questions: agent routes went from 8 of 10 to 10 of 10, correct answers demoted to a guess from 2 of 10 to 0 of 10, and the three refusals held on both sides. The blind abstention set over the same demo, 50 questions: 28 of 30 out-of-scope questions not answered confidently on either side, the same two medical baits through at 33.0 and 24.8, above either floor, and the in-scope column moved from 6 confident, 8 guess and 6 nothing to 12, 2 and 6. Nothing left the nothing column, because a question sharing no token with the corpus scores zero at any floor. The private fleet of 226 entries, the one the constant was calibrated on: identical line for line, neutral by construction and checked rather than assumed.

$ kb eval examples/demo/gold.tsv examples/demo
gold:     13 questions, 10 answerable
indexed:  11 entries across 3 agents

  file agent kw    score  marg verdict question
  ok   ok    ok     33.7  4.50 hit    quem decide se um deploy pode ir pra producao
  ok   ok    ok     11.2  1.00 hit    como faco rollback sem downtime
  ok   ok    ok     22.5  1.00 hit    devo usar sqlite ou postgres no comeco
  ok   ok    ok     29.3  1.00 hit    what makes a review comment blocking
  ok   ok    ok     24.8  1.00 hit    como prender a atencao nos primeiros segundos do video
  ok   ok    ok     24.8  1.00 hit    qual a melhor frequencia de postagem
  ok   ok    ok     66.9  5.95 hit    how much protein per meal
  ok   ok    ok     11.2  1.00 hit    quanto tempo um musculo leva pra recuperar
  ok   ok    ok     24.8  1.00 hit    onde consulto a tabela nutricional de um alimento
  ok   ok    ok     33.0  1.00 hit    quantos gramas de proteina por quilo
  ok   -     -       0.0  0.00 none   qual acao da bolsa devo comprar
  ok   -     -       0.0  0.00 none   me conta uma piada
  ok   -     -       0.0  0.00 none   qual o sentido da vida

FILE   fused   10/10  (100%)
       keyword 10/10  (100%), the same question asked of the keyword scorer alone
AGENT  routes  10/10  (100%), ...
       keyword 10/10  (100%), ...
       fused   10/10  (100%), ...
       always-yaron   4/10  (40%), the best a fixed choice can do on this set
       routing beats the fixed choice by +6 questions

GATE   flagged 0/0 of its own misses as a guess
       demoted 0/10 correct answers to a guess
       of 3 question(s) the set says to decline: refused 3, hedged 0, answered 0

SPEED  1779 us per question fused, 710 us keyword only. No model, no network.
       Release binary.

The same verdict, for a program. kb route --json carries the floor that applied beside the score it was measured against, so the gate can be argued with rather than guessed at, and a caller branches on gate.served and never on the length of results, because a refusal still carries its candidates.

$ kb route "quem decide se um deploy pode ir pra producao" examples/demo --json
{"agent":{"contenders":2,"files":1,"margin":4.5,"name":"zed","score":33.69244,"totals":[{"agent":"zed","score":33.69244},{"agent":"steve","score":7.487209}]},
"confidence":{"agreement":2,"keyword_score":33.69244,"margin":4.5},
"gate":{"floor":6.916209,"ranked_by_text_only":false,"served":true},
"index_was_rebuilt":false,
"indexed":{"agents":3,"aliases":0,"entries":11},
"keyword_top":"zed/knowledge/deploy-checklist.md",
"miss":null,
"question":"quem decide se um deploy pode ir pra producao",
"results":[ ... ],
"skipped":[],"suggestions":[],"verdict":"hit"}

The tool failing its own test

On the private fleet's 33-question key the GATE block ends in OVERLAPS. In the run the README quotes, hits score 29.43 to 129.87 and misses 7.45 to 91.86, so no single floor separates the two, and the gate flags only 1 of its 13 misses as a guess. What it does not do is demote a correct answer, 0 of 11, and it declines 4 of the 9 questions the key says to decline. The decision record of 2 September 2026 reports a later run of the same key, hits 33.15 to 145.95, misses 7.98 to 93.60 and 3 of the 9 refused, which is the same overlap with the figures moved. The README keeps the block because it is the tool failing its own test in public; an earlier version quoted a run that separated cleanly, over a 19-question set that no longer exists.

$ kb eval fleet/zed/fleet/eval/gold.tsv .

  GATE   flagged 1/13 of its own misses as a guess
         demoted 0/11 correct answers to a guess
         abstained on 4/9 question(s) the set says to decline
         hit scores  29.43 to 129.87
         miss scores 7.45 to 91.86
         OVERLAPS: no floor tells a hit from a miss on this set.

What the floor still does perfectly is tell nothing from something: a question that shares no token with the base scores zero and abstains, which is the whole of the three refusals above. What it cannot do is tell a real match on the wrong file from a real match on the right one. That job is handed to the model described further down, and it is the reason that model exists.

Two things reopen the floor: the first fleet that misroutes at two to four entries, which is the number MIN_ENTRIES_TO_ROUTE is waiting for, and the first fleet near a thousand entries, where the large side of the ledger stops being arithmetic. kb eval on it is the instrument, and gate.floor in every JSON reply says what applied. The decision record behind this section is retold in the writing at The confidence floor now scales with your corpus.

What a miss returns

When nothing clears the floor the librarian does not hand over the least wrong book. She says she does not have it, and then she says what she does have that looks like what was asked: the base's own vocabulary, chosen by character trigram overlap between the question's words and the keys on every label, at most eight terms, in microseconds, with no dependency. Trigram overlap measures spelling. It reaches a typo and a cognate across languages and it never reaches a translation, and the reply says which kind of help it is, because a suggestion whose limits are not stated gets read as the whole answer. When no orthographic neighbour exists it offers nothing rather than the nearest thing it holds. That property has its own test, because a suggester that always answers is one nobody can use.

$ kb route "come funziona il registro delle decisioni" examples/demo
question: come funziona il registro delle decisioni
indexed:  11 entries across 3 agents, 0 aliases

  nothing matched. Either the base does not cover it, or the
  Search for lines do not carry the words a real question uses.

  the base does know these, and they look like words you used:
    technical decision
  that is spelling and not meaning, so it finds a typo or a cognate
  and never finds a translation.

decisioni reached technical decision, a key on the demo architect's own index, with no alias line, no model and no dependency. The same reply, where spelling misleads:

$ kb route "how do I roll back" examples/demo
question: how do I roll back
indexed:  11 entries across 3 agents, 0 aliases

  nothing matched. Either the base does not cover it, or the
  Search for lines do not carry the words a real question uses.

  the base does know these, and they look like words you used:
    backup do banco, database backup, parar o scroll, stop the scroll
  that is spelling and not meaning, so it finds a typo or a cognate
  and never finds a translation.

The book that answers this question is keyed rollback, one word, and the question wrote it as two. roll reached scroll and back reached backup, four terms spelled near and none of them meant. A reader sees that at once from a reply that has said what kind of help it is. A scorer cannot see it, and this one does not pretend to.

Why not a model on the miss path

The specification for this step named a local model rewriting the question into candidate terms, 20 to 40 tokens, the cheapest kind of local generation there is. It was costed before it was built, on 2026-08-17, from generation measured on the author's Dell Latitude 3420, an i5-1135G7 with four cores and no dedicated GPU, at 5.55 to 5.88 tokens per second: that puts the expansion at 3.4 to 7.2 seconds, against 4.43 ms for the whole of kb_retrieve end to end, both figures from the same machine. Roughly a thousand times the retrieval, paid on the path where the question already failed, so the person waits five seconds to be told the library may not cover the subject. The trigram half shipped first because it costs nothing and can state its own limit exactly. The model half stays specified and unbuilt, and the miss log decides whether it is ever needed.

The log, and what a miss earns

Every refusal is written to kb-misses.txt beside the fleet manifest: one line per distinct question with its count, first seen, last seen and the question verbatim, and an indented looked like: line recording what the base offered back, so it is visible whether the free suggestion helped or a translation was needed. The count is the priority; a question that missed five times earns its fix before one that missed once. It is a recall loss log, not a list of chores, and it is private by construction: these are a person's real questions, a nutrition agent's miss is a health question, so the file is gitignored wherever it lands. Over kb route --json the same loss comes back as the miss object, with the question, the words offered, the date, the path the log was written to and whether the write succeeded, so a deployment whose filesystem cannot keep the file can keep the loss.

What a miss eventually earns is an alias line in kb-aliases.txt, alias = canonical, one per line. Expansion is additive, the original words always survive, so a wrong alias can add noise and can never remove signal. Only add a line after a real question missed: it is a record of misses and not a dictionary, and a dictionary is what makes it unmaintainable. It is also how a fleet answers questions in one language over a base written in another, which is the loud case; the quiet case is the same shape and more common, a file keyed sono and a person who said dormir.

Writing a label

The Search for line is the whole ranking, and the most common way to own a base that holds the answer and cannot find it is to fill that line with topic words. Write the keys as questions, the phrases somebody types from inside the problem, not as subjects. The note carries its own keys, directly under its title, and a file is in the index if and only if it declares them; a map file, when a base keeps one, is a reading list for people, and the catalogue does not consult it. Three questions against one five-file base, as they printed under the fixed floor of 17.5:

"quanto custa o frete"                             hit     20.68   matched: quanto custa o frete, frete
"o frete e gratis a partir de quanto"              hit     19.24   matched: frete gratis, frete, gratis
"quanto tempo demora pra cair o estorno no cartao" guess    6.19   matched: tempo, estorno

The third note is not worse written. Its keys are reembolso, estorno, devolucao do dinheiro, which name the subject correctly and match no phrase anybody types. Under the floor a five-file base gets today, 4.6, that third line is a hit at 6.19 rather than a guess: right file, a third of the score. That is arithmetic from floor_for(5) and not a re-run, because the base is not in the repository. The point about phrases stands either way, and it is the point.

kb check grades the line, and two of its warnings are the ones to act on first. W06, thin keyword line, fires under twelve terms and asks for thirty, in both languages, including the words somebody types from inside the problem. The number comes from a real miss: a file keyed eating out, restaurant, poke, salmon scored zero on a real question and, widened to thirty terms, answered it at 130.21. W07, unsearchable key, fires when a written key collapses to one word after stopwords, so it reaches neither index. The example that earned the check is nao contestar indexed as contestar, its own opposite, until somebody rewrote it as proibido contestar.

Who chooses the agent, and what it is shown

Which book holds the answer and which agent should give it are different questions. Finding the book is a lookup, lexical, and the catalogue answers it exactly. Choosing the agent is a judgement about a subject, and no count of shared words makes it: ask how to deploy with zero downtime and monitor the infrastructure, in a fleet where nobody operates systems, and every base scores zero or noise, while a reader who knows only that one agent builds software and another does marketing places the question at once, and can also say the thing that matters most, that nobody here covers it. A score of zero says no match. It can never say no one here does this kind of work.

So the one model in the path sits here, after retrieval and never inside it. Retrieval stays deterministic, reproducible, and the only thing that reads the books. What reaches the model is a dossier: the roster, with what each agent does and where it stops; the question; up to five lines of evidence as base/path with the keyword score and the words that matched; one about: line per file, taken from the note's Exists to line, because a path is not evidence about a subject; and a per-agent table of the best score in each species, memory, skills and tools, with a dash where there is none. Roughly three hundred tokens, and never the base. It cannot invent a file, because it never sees one, and its answer is a name from the list it was handed; a name off the roster resolves to no owner.

Coverage is a first-class answer with three values: covered, adjacent and uncovered. Adjacent names the nearest agent and says plainly that answering from there is a stretch, and that the honest options are to give that agent the knowledge or to create an agent for the subject. Uncovered prints one line, the same on the desk, on the hook and in the reading room, so one wrong answer cannot be worded three ways:

VESTA: nothing in this fleet covers {subject}, and no agent is near enough to name. {reason} This is a gap in the fleet rather than a gap in the question: it is worth deciding whether an agent should exist for it.

The command is named in fleet.txt, on the line classifier =, and it is a process rather than a provider: the dossier on stdin, the verdict on stdout. Any model behind any runtime satisfies that, including a local one, and kb gains no dependency and no network code for it. With no line configured, or a command that fails, the deterministic fold described above chooses, so the fleet never stops answering because a model was unavailable.

The price, with its conditions: 9 to 15 seconds per message through the Claude Code CLI on the author's laptop, the figure the decision record of 2026-08-19 carries after its correction of 2026-08-20. That line read 13 to 16 until 2026-08-20, and the real figure then was 43 to 48 seconds against a hook budget of 30: the process had been spawned with its working directory set to the fleet root, so the CLI inspected 11,510 files and 2.5 GB of build output on every message before reading a dossier that arrives whole on stdin. Starting it in an empty directory outside the fleet fixed it. The way down is a faster chooser, a local model or a resident process, and not a cheaper decision about when to think. Local model routing is not built.

Why not a score, once more. A cascade that called the model only when the deterministic choice did not dominate its field was built and rejected inside the same decision. It cut the common case from about 14 seconds to about 1, and it routed that same deployment question to marketing in 971 ms, because the word zero matched three of the marketing agent's research notes and gave it the whole of a field nobody else was in. A cascade can only gate on the deterministic score, and that score does not know when it is wrong: one agent alone in the field does not distinguish plainly theirs from a coincidence of vocabulary, since those are the same number. A gate built on a blind signal inherits the blindness, so no threshold rescues it. The constant and the measurement are kept in the source so the idea is met rather than rediscovered.

One contract, every surface

kb is a library as well as a binary, and Memory is the contract: three verbs over a set of bases, route, retrieve and remember, and the one place an answer is computed. The CLI and the reading room call it, kb serve wraps it in MCP for other people's runtimes, and the tray app links it as a library, so no two surfaces can answer differently about one question. Reaching past it into the modules means rebuilding the pipeline, which is how two bugs happened: alias expansion reaching one scorer and not the other, and the two oversampling factors drifting apart. mcp.rs assembled the pipeline by hand for one commit, and that is exactly why it does not any more.

The same rule holds for what counts as a miss. Memory::recall_loss decides whether a question was a recall loss, records it, and hands back the vocabulary to offer, in one call. Before 2026-08-30 six surfaces decided it four different ways: two tested the length of the keyword list, two the fused list, and kb answer suggested vocabulary without ever recording anything. The surface a deployment used had the hole in it, so the loss most worth having, a question the text scorer answered and the gate refused, went unrecorded everywhere it mattered. A guess is not a loss: it was served, with a warning, and that line is pinned by a test because moving it changes what the log counts.

A path handed to any verb may be a base, or a fleet root, a directory that is not itself a base but whose immediate children are. Both work and neither is required, because requiring an arrangement would be an assumption about the reader's filesystem. A directory holding a map of its own is a base and is never expanded, even when it contains bases, since expanding it would silently drop the parent's own notes.

What was measured and kept out

None of this is a position against models. It is a set of measurements the models lost, on this corpus, at this scale, by the system's own code. BGE-M3, on 2026-08-17, against twenty real questions on the private fleet, embedding the note bodies chunked the way kb chunks them: 13 right against roughly 16 for the keyword scorer. Losing the direct comparison is not what disqualified it, because fusion does not need a second scorer to win, it needs it to be wrong about different questions. So the fusion was computed directly. Nineteen of twenty answers were unchanged; the one that changed moved a correct answer onto an unrelated file, and the question the keyword scorer honestly did not answer stopped returning nothing and started returning another unrelated file. An honest abstention became a confident wrong answer, which is the failure the whole system exists to avoid. It also cost 2.2 GB and 2,833 seconds to index 1,039 chunks, 47 minutes at 2.7 seconds per chunk.

Two cross-encoder rerankers on 2026-08-18, over the 19 answerable questions of the same set, each handed the keyword scorer's top five with the right file first 16 times: bge-reranker-v2-m3 returned it first 11 times at 2,571 ms per pair, and jina-reranker-v2 14 times at 647 ms per pair, under a non-commercial licence that would have kept it out of a product regardless of quality. Both degraded the ranking they were handed. A sceptic commissioned against the reranker plan predicted zero marginal gain before the numbers existed, and the reality was negative.

The finding that outlives the decision is about scores rather than about any model. The keyword scorer's one wrong answer on that set scored 3.82 while every correct answer scored 9.55 or more. BGE-M3's correct answers scored 0.496 to 0.659 and its wrong ones 0.510 to 0.608, overlapping completely, and no model measured that day produced a usable abstention signal. The keyword scorer is the only scorer whose score separates a hit from a miss, and that is what makes the floor a mechanism at all. A dense scorer that never abstains removes the property that separates this library from the ones that always return a top hit.

Three things reopen it: a base past roughly a thousand files, where hand-written keys plausibly stop covering it and a dense scorer's advantage grows; a question set nobody tuned against showing the keyword scorer below the dense one; or an embedding or reranking model under a permissive licence and under 1 GB that separates a hit from a miss on somebody else's benchmark, the one property no candidate had. The instrument is tools/bench, one Rust command, and every model artefact was deleted after the review, so re-running it downloads them again.

Previous: Citing What You Read · Next: On Your Machine · All of the docs · Back to the library