Phrase alignment re-embeds every wide signature on every run · Issue #1097 · deeplethe/utopia · GitHub
Skip to content

Phrase alignment re-embeds every wide signature on every run #1097

Closed

Description

@evolv3ai

Problem

On every align_phrases run, phrase_alignment::shortlist embeds the text of every live "wide" signature in the base: every signature with more than SHORTLIST (10) structural candidates. It doesn't embed only the new or changed ones. It sends batches of SHORTLIST_EMBED_BATCH (32), one at a time, before the run decides anything. Nothing caches the vectors between runs, so a run that has two new signatures to decide still re-embeds every unchanged signature in the base first.

This is still the case on dev (ee3d1e2), in shortlist() in phrase_alignment.rs.

There are two consequences:

  1. The per-run floor grows linearly with the base. We measured 16–19 ms per wide signature, which is the embedding round-trip with batches in series:

    • ~4.4k wide signatures: 69–103 s per run;
    • ~17k wide signatures: ~270 s per run (claim → 候选短名单开好).

    A base with ~700k wide signatures would spend about 3 h embedding on every run. That cost repeats for every small incremental batch of documents, and it's paid in embedding calls each time.

  2. When the base's properties have no vectors, every one of those embeddings is discarded. nearest_relation_type_ids returns nothing and the loop continues for every signature, so the run logs shortlisted=0 after embedding the whole set. On rc7, a base's relation types only get vectors when embed_ontology runs, which happens when the workspace's embedding settings are saved. So a base created, or given its properties, after that save has no property vectors, and pays for the embeddings without getting a shortlist. Refresh the ontology vector index when the editor creates or relabels a class or property, before alignment runs #956 on dev fills vectors when the editor creates or relabels a property, which narrows this case but doesn't remove it.

Repro (generic)

  1. Configure a workspace with an embedding model, then create a base and add more than 10 relation types with no domain or range. Most signatures are then "wide".
  2. Ingest some documents so align_phrases runs.
  3. Ingest one more short document. The next align_phrases run logs 候选短名单开好 wide=N shortlisted=0. That N is every wide signature in the base, not the few new ones. Claim → shortlist time scales with N.

Suggested fix

  • Cache signature vectors. Key them by embedding model plus a hash of the embed text (phrase · example · quote), so a run only embeds signatures whose text it hasn't seen. The text of an unchanged signature is stable: its examples are the first three by recorded_at, id.
  • Skip the embedding when it can't be used. If the base has no relation-type vectors of the needed kind (relation or attribute), check that once with an EXISTS and don't embed at all. That matches today's result, an empty shortlist, without the calls.
  • shortlist_kind_words has the same shape, so the same treatment probably applies there.

Either change keeps the fingerprints, and therefore the decisions, unchanged. A vector cache only removes repeat work.

Activity

  1. WaylandYang commented on Oct 8, 2026

    @WaylandYang
    Contributor

    Thank you, confirmed in shortlist().

    Please take it in two cuts:

    1. Skip when it cannot be used. One EXISTS for property vectors of the needed kind before any embedding, in shortlist and shortlist_kind_words. No schema change.
    2. Cache the vectors. A table keyed by base, embedding model and a hash of the embed text, holding a dimensionless vector like chunks.embedding. A run embeds only unseen texts and deletes rows whose hash no longer occurs. It takes migration 0107, and CURRENT_SCHEMA_VERSION becomes 98.

    For cut 2, please include a test that a second run over an unchanged base makes no embedding call.

    Thanks again for both reports.

  2. WaylandYang commented on Oct 9, 2026

    @WaylandYang
    Contributor

    Cut 1 landed in #1107 (5be57cb on dev): signatures and kind words are not embedded while no property has a vector. This issue stays open for cut 2, the vector cache. Thanks, @evolv3ai.

  3. WaylandYang commented on Oct 10, 2026

    @WaylandYang
    Contributor

    Thank you for the report and for both cuts. With #1107 and #1186 on dev, signatures are not embedded while no property has a vector, and once they are, a run embeds only the texts it has not seen. Both will be in the next release. Closing this as done; please reopen if a run on your base still spends its time embedding. Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions