fix(search): use global stats for shard-independent text scoring (#7250)
commit 1e0e162 ↗ · dragonflydb/dragonfly · · committed May 20, 2026 · +619−51 · base 460bd5d2b522
Agent prompt
what a new run launched now would send
FT.SEARCH / FT.AGGREGATE text scoring is shard-dependent: BM25STD, TFIDF, and TFIDF.DOCNORM compute IDF and average field length from each shard’s local documents, so the same indexed corpus and query can yield different top-K orderings and scores when the process runs with one proactor thread versus several.
Reproduce with a diverse TEXT corpus (~200 docs), FT.CREATE on HASH, then FT.SEARCH with WITHSCORES + SCORER (and separately FT.AGGREGATE with SCORER + ADDSCORES + SORTBY @__score DESC) under proactor_threads=1 and proactor_threads=4. Compare top-K keys and scores.
Acceptance criteria:
1. For a fixed corpus and query, FT.SEARCH WITHSCORES+SCORER top-K keys and scores are identical (scores within small FP tolerance) regardless of shard/proactor count, for BM25STD, TFIDF, and TFIDF.DOCNORM.
2. Same invariance for FT.AGGREGATE with SCORER+ADDSCORES+SORTBY @__score (stable tie-breaking when scores tie).
3. FT.SEARCH with SORTBY on a sortable field plus WITHSCORES must keep keys, ranks, and scores consistent — score-oriented reordering must not desync SORTBY ordering.
4. Single-shard behavior and non-scoring / KNN-driven paths should remain correct.
Work only inside this repository checkout. Make the code change the task
describes, keeping the diff focused — no drive-by refactors.
When you are done, leave your changes committed or in the working tree;
they are collected automatically.
Stay on this snapshot checkout (`task/ycb_dragonfly_c1e0e162d4ef3`). Never checkout, pull, or rebase onto `main`. That branch is a README-only orphan.
Stay on this HEAD. Do not fetch another default branch. Push only on the Cursor-created `crazy-cursor/…` side branch from this HEAD.
Some past runs of this task were launched with a different prompt (the prompt template changed since, or those runs predate this benchmark's stored prompt). Each run persists the exact prompt it sent at launch — that per-launch record is the audit trail; this page shows only the current one.
Reference diff
The task commit's diff against its first parent — the judge's comparison target.