{{ message }}
Repository navigation
Choosing the mappings and searching chunks reuse the question's embedding instead of embedding it again in the same turn - #978
Merged
WaylandYang merged 1 commit intoSep 27, 2026
Conversation
…ding instead of embedding it again in the same turn Signed-off-by: dada-yan <BinjunYann@gmail.com>
WaylandYang
approved these changes
Sep 27, 2026
WaylandYang
left a comment
Contributor
There was a problem hiding this comment.
Thank you. One cache for the turn, created before the mappings are chosen and handed to the sink, and the callers without a turn pass None. A cache hit still reads the settings for the model name in the mapping search, which is the right caution. Landing it. Thanks again.
WaylandYang
approved these changes
Sep 27, 2026
WaylandYang
left a comment
Contributor
There was a problem hiding this comment.
Maintainer review: sound, landing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

The follow-up suggested in #971's review:
search_chunksandmapping_index::relevantshare the turn's embedding cache.Problem
ToolSink.mapping_index::relevant, before the first model request) and the vector channel ofsearch_chunks(retrieval::hybrid). A question over a base with a data source was embedded up to three times per turn: once for the mappings, once for the search when the model passes the question as its query, once for the first lookup by name.Change
llm_util::EmbedCachenames the cache the sink already held (HashMap<String, Vec<f32>>).retrieval::hybridandmapping_index::relevanttakeOption<&mut EmbedCache>. Their vector channels read the vector from the cache before reading the model settings, and put a newly embedded vector in it. A cache hit inrelevantskips the embedding request and the embed permit; it still reads the settings for the model name, since the mapping search compares only rows embedded by the same model.search_chunksand the graph lookups.search_chunkspasses the sink's cache. The HTTP search route, the mappings page and the one-shot RAG fallback passNoneand embed as before: they have no turn.Verification
Run in containers on a Linux host: Rust 1.98.1 and PostgreSQL 16 with pgvector.
Tests, in
chat_embed_once_tests.rsa_chat_turn_embeds_its_question_oncenow scriptssearch_chunkswith the question as its query beforefind_entitiesandentity_facts: three steps, one embedding request.mappings_search_and_the_graph_tools_share_one_embedding:relevanton a base with no confirmed mappings embeds the question once;hybridwith the same cache embeds nothing and still runs its vector channel;find_entitieson a sink holding the cache picks the entity closest to the question without embedding; a different search query is embedded once and cached; the same calls withNoneembed again.On dev (0fbcd38) with the new tests
The control is dev plus the chat-level test (the function-level one needs the new parameters and does not compile there).
a_chat_turn_embeds_its_question_oncefails there: the embedding service receives two requests, the search's and the lookup's, where one is expected.a_turn_embeds_its_question_and_each_word_oncepasses there as here (logs 520–522).This branch
The embedding tests (10),
api::withretrievalandmapping(213),cargo fmt --all --check,cargo clippy --locked --workspace --all-targets -- -D warnings,UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspacewithUTOPIA_TEST_REQUIRE_DB=1(1,228 passed, 0 failed, 5 pre-existing ignored) andcargo build --locked --workspaceall pass (logs 523–530). No web file changes.Also on a Mac with the same toolchain and a scratch PostgreSQL 16: the embedding tests (10),
api::,retrievalandmapping(213), fmt and clippy clean.Known limits
queryis the question verbatim; a rephrased query is embedded once and cached for the rest of the turn.