feat(rerank): rerank by a decision model with rerank.decision by jfrancoa · Pull Request #13528 · weaviate/weaviate · GitHub
Skip to content

feat(rerank): rerank by a decision model with rerank.decision - #13528

Draft
jfrancoa wants to merge 4 commits into
jose/decide-openaifrom
jose/rerank-decide
Draft

jfrancoa wants to merge 4 commits into
jose/decide-openaifrom
jose/rerank-decide

Conversation

@jfrancoa

@jfrancoa jfrancoa commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What's being changed:

rerank by a decision model: Rerank.decision on the gRPC request sends the rerank query to the collection's decisions module as a yes/no question about every candidate. The probability is the rerank_score, candidates below min_probability are dropped, and order keeps the search order or sorts by probability. The RFC section is "rerank with a decision model".

Proto. Rerank.decision (RerankDecision { optional double min_probability = 1; optional RerankOrder order = 2; }, RerankOrder search or probability). A field left unset takes the class setting of the module (minProbability, order, on both decisions-typesafeai and decisions-openai); a value out of range fails before the search.

Routing (usecases/modules). A class may have one reranker and one decisions module (#13477). rerank without decision goes to the reranker; a class with only a decisions module serves it as a relevance predicate. rerank with decision goes to the decisions module, and is an error on a class without one. decide always goes to the decisions module.

Shared rank provider (usecases/modulecomponents/additional/rank). Params.Decision carries the request values; DecisionDefaults reads the class settings; after Rank, the threshold and the order apply for decisions providers only, so rerankers are unchanged. The page can come back short: nothing is fetched beyond offset+limit (no over-fetch in this version, see the RFC), and maxDocuments bounds the page.

Modules. decisions-typesafeai loses its own filter (filter.go), the rubric settings (scoreLevels, minScore) and the per-request headers X-Typesafeai-Min-Probability, -Order, -Score-Levels, -Min-Score; threshold and order come from core. decisions-openai gets the minProbability and order class settings. decisions-dummy ranks by the share of the query words in the text, so the acceptance tests can check thresholds and orders.

Tests. Rank provider (threshold, order, class defaults, request override, rerankers untouched), modules provider (routing, the error without a decisions module), gRPC parsing (the sub-message, unset fields, out-of-range values). Acceptance (test/acceptance/decisions): class defaults, threshold, order by probability, the plain rerank fallback, class setting overridden by the request, the error on a reranker-only class, the out-of-range error, and a class with both modules.

Stacked on #13485. The rubric rerank (levels, min_score) is a later PR. Client: weaviate/weaviate-python-client#2198.

Review checklist

  • Documentation has been updated, if necessary. Link to changed documentation:
  • Chaos pipeline run or not necessary. Link to pipeline:
  • All new code is covered by tests where it is reasonable.
  • Performance tests have been run or not necessary.

@orca-security-eu orca-security-eu Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

@jfrancoa
jfrancoa force-pushed the jose/rerank-decide branch 2 times, most recently from ba924f7 to a389b0a Compare October 13, 2026 13:21
@jfrancoa
jfrancoa force-pushed the jose/decide-openai branch 2 times, most recently from 92bde34 to 82d568a Compare October 13, 2026 13:51
The Rerank gRPC message gains an optional decision sub-message with
min_probability and order. Its presence routes the rerank to the class's
decisions module, which answers the query as a predicate about every
candidate; the probability is the score, the candidates below the
threshold are dropped, and the survivors keep the search order or are
sorted by probability. A class without a decisions module rejects it, and
a plain rerank on a class with a reranker and a decisions module goes to
the reranker. The class settings minProbability and order of a decisions
module are the defaults the request overrides.

The threshold and the order live once in the shared rank provider, for
every decisions module. decisions-typesafeai loses its own filter, the
rubric on rerank and the X-Typesafeai-Min-Probability, -Min-Score,
-Score-Levels and -Order headers; a rubric is a score question of decide.
decisions-openai gains the minProbability and order class settings.
decisions-dummy answers a rerank as the predicate it already answers, so
the acceptance tests can tell the reranker from the decisions module on
one class.

[api]
A rerank by a decision on a class without a decisions module, or with a
limit above the module's maxDocuments, now fails in the gRPC parser as
decide does, before the search runs. The limit check is shared with
decide.
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant