fix: apply Neo4j invalidation filters before fulltext result limits by Frankie-Xu · Pull Request #1970 · getzep/graphiti · GitHub
Skip to content

fix: apply Neo4j invalidation filters before fulltext result limits - #1970

Draft
Frankie-Xu wants to merge 1 commit into
getzep:mainfrom
Frankie-Xu:fix/neo4j-temporal-fulltext-limit-1957
Draft

Frankie-Xu wants to merge 1 commit into
getzep:mainfrom
Frankie-Xu:fix/neo4j-temporal-fulltext-limit-1957

Conversation

@Frankie-Xu

Copy link
Copy Markdown

Summary

Neo4j can discard the current fact before an invalidation filter sees it. queryRelationships currently returns only the first 20 index hits for the hybrid candidate search, then Cypher applies invalid_at filtering. With 20 or more better-ranked superseded facts, a current fact below the vector threshold can still be missed.

Omit the Neo4j procedure-level cap for edge searches with an invalid_at filter, then keep the existing ordered result limit after filtering. Apply this to the production generic search path and the Neo4j operations path. Unfiltered searches and other providers keep their existing query behavior.

This is a focused follow-up to #1957; it does not include or replace that PR's candidate-filter change. The candidate filter in #1957 and this retrieval fix address different stages of the same failure. This PR alone does not fully resolve #1956; merge it together with #1957. The combined fixture tests exercise both changes.

Type of change

  • Bug fix
  • Feature (linked Feature issue already has rfc-approved)
  • Documentation
  • Maintenance or refactor

Related issue

Fixes #1956

Related candidate-filter PR: #1957, reviewed at de2c8a2c81cf697342c74293042510deee0d6f44.

Design approval

  • This is not a feature pull request.
  • The linked Feature issue has the rfc-approved label after discussion with the Graphiti team.

Testing

  • Tests were added or updated for behavior changes.
  • make check passes.
  • Tests are not applicable; the reason is explained below.

Local, no model/API calls:

  • New tests/utils/search/test_temporal_fulltext_candidates.py: 15 passed in an isolated checkout environment, Python 3.14.6. Cases cover 19/20/50 obsolete hits, exact cutoff, later-ending facts, 35 eligible facts still returning 20 in score order, unfiltered index-limit preservation, and unchanged other-provider helper output.
  • New tests plus existing edge-resolution module: 24 passed using existing dependencies read-only, with imports verified to come from this checkout. One existing Pydantic configuration deprecation warning.
  • In-process restoration of the previous procedure cap: 6 failed, 2 passed in the relevant new tests. The 19-obsolete-hit controls pass; 20/50 hits and later-ending candidates hidden behind 50 obsolete hits fail on both paths.
  • Exact fix: search invalidation candidates only among facts the new fact can affect #1957 head with a deterministic ranked-corpus executor: 12 passed, 2 failed; both failures were the newly identified BM25-only current-fact case. Applying this focused patch alongside fix: search invalidation candidates only among facts the new fact can affect #1957: 18 passed, including missing valid_at, cutoff boundaries, backfill and duplicate-start controls.
  • Changed-file Ruff lint and formatting pass. Pyright on the three changed core files passes with 0 errors and 0 warnings, using the read-only existing dependency environment as its interpreter. Independent code/spec review found no actionable defects; the reviewer independently ran the original 13 new cases before the two final result-limit controls were added.

The synthetic executor evaluates emitted WHERE predicates in SQLite while preserving the procedure-cap/filter/final-limit order. The combined fixtures execute Graphiti's query construction, record parsing, hybrid RRF and resolver with a stub judge. These are not native Neo4j, Lucene ranking, embedding-quality or model-accuracy measurements. The optional procedure options map is documented by Neo4j.

Native Neo4j integration, performance, Python 3.10, and full make check have not been verified. No Docker daemon is available locally.

Breaking changes

  • This change affects an existing public API, data model, or deployment.

Existing helper callers retain the default behavior. Searches with invalid_at filters can visit more fulltext hits; their final ordered result limit remains unchanged. This does not fix ranking among more than 20 eligible facts, duplicate/other-field search limits, or the separate Kuzu/Neptune pre-filter candidate caps.

Checklist

  • I opened or linked an existing issue before this pull request.
  • I reviewed my own changes.
  • I updated documentation for changed behavior or public interfaces.
  • I did not commit secrets, credentials, customer data, or other sensitive information.
  • I will complete the Contributor License Agreement when prompted.

AI assistance: implemented and reviewed locally with OpenAI Codex.

@zep-cla-assistant

Copy link
Copy Markdown
Contributor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Edge invalidation misses the current fact once a subject has ~10 superseded facts

1 participant