Problem

With reranking enabled, a hybrid request whose result window exceeds the fixed reranking prefix performs full-text and vector retrieval twice. The second pass reconstructs a stable candidate pool for pagination. Reranking itself runs once.

Production Cloud 37d681c (Core fa30662) reproduced this with search_notes(query="tenant routing search architecture and memory usage", search_type="hybrid", search_all_projects=true, page_size=25) across 17 projects.

Evidence

Logfire project basic-memory-cloud, production trace 01a0a2f7674171c89b468c1eb6970d76, 2026-09-15 02:48:37–02:49:37 UTC:

• Server request 59.108s; project resolution 6.810s, followed by overlapping project searches through 58.601s.
• 34 FTS spans: 53.884s cumulative, 10.298s maximum.
• 34 query embedding spans: 24.863s cumulative, 7.328s maximum.
• 34 vector-query spans: 23.511s cumulative, 4.949s maximum.
• 17 rerank spans: 216.825s cumulative, 20.591s maximum.

Cumulative durations include concurrent work and must not be added as request wall time. Reranking is the largest cost; duplicate retrieval is an additional independently observed inefficiency. The repeat hit all 17 caches and took 10.762s client-side. This is not a controlled benchmark.

Code path

In src/basic_memory/repository/search_repository_base.py, _candidate_limit() expands the retrieval window beyond _rerank_candidate_limit() when the requested window exceeds _reranker_candidates. _search_hybrid() then recursively calls itself with the fixed candidate limit and _apply_rerank=False. This repeats FTS, query embedding and vector retrieval before the single _rerank_and_paginate() call. Inspect the analogous vector stable-pool path too.

The fixed pool protects pagination: expanding the retrieval universe can alter fused scores and must not reorder a previously returned prefix. Removing the recursion without preserving that invariant is not sufficient.

Desired outcome / acceptance

• Avoid duplicate retrieval work for expanded hybrid/vector windows while preserving fixed reranked-prefix membership, deterministic tail ordering and page completeness.
• Reuse query embeddings where the query/model are identical within the request.
• Cover first page, larger page sizes, deeper pages, chunk-to-document collapse, mixed FTS/vector matches and reranking disabled.
• Regression tests must demonstrate the duplicate work before the fix and fail if stable pagination breaks. Exercise real SQLite and Postgres retrieval and API/MCP pagination boundaries.
• Use stage spans to compare retrieval call counts and latency; distinguish inference execution from queue waiting if tuning reranker performance separately.
• No cross-worker semaphores or admission-limit system in this issue.

Cloud is disabling production reranking as an operational mitigation. Existing bounded previews and ONNX batch/arena tuning remain in place. No implementation is included in this issue.