{{ message }}
Repository navigation
Conversation
…anks and source, and pages, so a backend can weigh and walk the results without a person's session Signed-off-by: Bogdan Szabo <bogdan@re-cinq.com>
This was referenced Oct 16, 2026
gedaiu
marked this pull request as draft
October 18, 2026 06:34
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Part of #1271. Related: #1268 (an
apipush keeps itsexternal_id, which the results now carry), #1262 (the graph reads a token gets).What happened
POST /kbs/{id}/searchtook a session only, returned chunks without scores or ranks, had no paging, and fell back to full-text silently when the vector arm could not run. A backend could neither call it nor judge what came back.What changed
utopia_search::rrf_fuse_scoredreturns each id with its fused score and its rank in every list (same k = 60, same tie-break);rrf_fusewraps it.retrieval::hybrid_scored(.., SearchOpts { top_k, offset, as_of }, embeddings)returnsSearchPage { results, has_more, vector_used }; each result addsscore,bm25_rank,vector_rank(1-based,nullwhen the arm did not find it),source_idandexternal_id(forapiandstatementssources). Each arm takesmax(offset + top_k + 1, 24)candidates, the extra one to tellhas_more.hybridwraps it with offset 0, so chat and tools are unchanged.documents::chunks_by_idsjoins the source and returnsSourcedChunk(ChunkViewflattened plus the two fields) instead of a second copy of the query.SessionOrTokenwithrequire_reader_kb(madepub(crate)in place, as in A token reads the entities of the bases it reaches #1263), acceptsoffset, keepstop_kat 50 or less, and refusesoffset + top_k > 100with 422page_too_deep. The response is{results, offset, top_k, has_more, vector_used}; every field it had is still there.SessionOrTokenlist inauth.rs,access-and-audit.mdand 0014 decision 6 name chunk search;chat-and-mcp.mddescribes the scored response and that an RRF score ranks within one query only. No ADR owns hybrid search, so no new record.Filters by source or filename come in the next part of #1271.
Tests
api::search_routes_tests, against a real database with five indexed chunks and no embedding model: a read token gets each hit with score1/61, ranks, source andexternal_id, andvector_used: false; pages of two cover the five hits without overlap andhas_moreturns false on the last; a page past the hundredth hit is a 422. With the route opened to tokens but without scoring and paging, each fails for its own reason.rrf_fuse_scoredhas a unit test of the exact scores and ranks.Run in
rust:1-slim-bookwormagainst Postgres withUTOPIA_TEST_REQUIRE_DB=1:cargo fmt --all --checkandcargo clippy --workspace --all-targets -- -D warningsclean;cargo test -p utopia-search7 passed;cargo test -p utopia-server -p utopia-store: utopia-server 665 passed, 2 ignored, utopia-store store suite 311 passed.