fix(extract): keep source_file repo-relative under a symlinked scan root (#4624) by maruf-asterbyte · Pull Request #4625 · Graphify-Labs/graphify · GitHub
Skip to content

fix(extract): keep source_file repo-relative under a symlinked scan root (#4624) - #4625

Open
maruf-asterbyte wants to merge 1 commit into
Graphify-Labs:v8from
maruf-asterbyte:fix/4624-symlinked-root-source-file
Open

maruf-asterbyte wants to merge 1 commit into
Graphify-Labs:v8from
maruf-asterbyte:fix/4624-symlinked-root-source-file

Conversation

@maruf-asterbyte

Copy link
Copy Markdown

What does this PR do?

Fixes #4624.

extract() resolves root, but _sf_entry, the relativizer for absolute source_file values, called sf_path.relative_to(root) on the unresolved path. When the scan path goes through a symlink, that call raises, and the file is treated as out-of-root, so source_file becomes the bare basename even with root= passed. This hits macOS projects under /var or /tmp (/var → /private/var) and any symlinked checkout.

Downstream code compares source_file as a string, so files that share a basename become one file. In a Next.js App Router project every route is page.tsx. A build_merge that prunes one deleted page.tsx then also prunes the freshly re-extracted nodes of another, and in the issue's repro the merged graph comes out empty.

The fix: before declaring a file out-of-root, retry relative_to with the parent directory resolved. This mirrors the #2169 remap pass in the same function, which already retries resolved. Only the parent is resolved. Resolving the file itself would follow a file symlink onto its target and merge the two sources that #2832 keeps distinct, and test_warm_cache_keeps_target_and_symlink_sources_distinct caught exactly that in an earlier version of this patch.

Type of change

  • Bug fix

Verification & Invariants

Invariant: an in-root file's source_file is its root-relative POSIX path, however the root was spelled, and two distinct source files never share a source_file.

  • Read the CONTRIBUTING.md guide.
  • Reproduced the issue and identified the invariant (see extract(): source_file collapses to basename when the scan root is reached through a symlink (macOS /var, symlinked checkouts) #4624).
  • Made the smallest fix necessary (one except branch in _sf_entry).
  • Added a regression test. All 3 tests in tests/test_symlinked_root_source_file.py fail on v8 @ b90b9df and pass here:
    • source_file is repo-relative under a symlinked root.
    • Two page.tsx files stay distinct.
    • The build_merge prune scenario keeps the re-extracted nodes.
  • Kept the PR description synchronized with the final implementation.
  • Documented limitations. A file reached through a directory symlink inside the root that points elsewhere inside the root is now recorded under the symlink target's directory, which is the same choice the Incremental --no-cluster extraction overwrites the full graph with the changed-file subset #2169 id-remap pass already makes. Files that really are out-of-root keep the existing _portable_out_of_root_sf behavior, because the retry fails the same way for them. Persisted state: graphs built under a symlinked root hold basename source_files, and the next rebuild rewrites them to full paths.

How was this tested?

uv run pytest tests/test_symlinked_root_source_file.py -q          # 3 failed on v8, 3 passed with fix
uv run pytest tests/test_cache.py tests/test_definition_file_portability.py \
  tests/test_source_key_memoization.py tests/test_stat_index_portability.py -q   # 100 passed (with the new file)
uv run pytest tests -q        # 8472 passed, 15 skipped, 1 failed
uv run ruff check .           # All checks passed
uv run pyright graphify/extract.py   # 135 errors, identical count on unpatched v8

The one suite failure, tests/test_skill_no_backtick_substitution.py::test_control_a_backticked_unknown_command_is_caught, also fails on unpatched v8 @ b90b9df on this machine and is unrelated. macOS, Python 3.11. The new tests skip when symlinks are unavailable, for example on Windows without developer mode.

Graphify-specific checklist

  • No skill fragments touched, so no skillgen --bless is needed.
  • AST/structural extraction stays deterministic. The new path only depends on the filesystem's own symlink resolution, which the remap pass already uses.
  • Reviewed for security. No shell or Python interpolation.
  • No API keys or local-only graph data included.
  • AI authorship is disclosed in the commit message (Co-Authored-By: Claude).

🤖 Generated with Claude Code

…oot (Graphify-Labs#4624)

extract() resolves `root`, but _sf_entry relativized each absolute
source_file against it UNRESOLVED. Under a symlinked prefix (macOS
/var -> /private/var, or any symlinked checkout) relative_to raised, the
file took the out-of-root branch, and source_file collapsed to the bare
basename, even with root= passed. Every page.tsx in a Next.js app then
read as one file, so an incremental merge pruning one deleted page.tsx
also pruned the re-extracted nodes of another, emptying the graph.

Retry with the parent directory resolved before declaring a file
out-of-root, mirroring the Graphify-Labs#2169 remap pass. Only the parent is resolved:
following a file symlink onto its target would merge the two sources
that Graphify-Labs#2832 keeps distinct.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; formal verification proved 0 of 1 changed function(s) (1 sampled, not proven).

Formal verification. PR-changed functions: 1/1 verified (0 proven, 1 may-equivalent, 0 distinguished) · 0 not verified.


Graphify review — findings

Fixes source_file collapsing to a bare basename when the scan root is reached through a symlink (macOS /var -> /private/var, symlinked checkouts), which made every page.tsx look like one file and let pruning one wipe another's nodes. When the raw path isn't under the resolved root, extract now retries with only the parent directory resolved before falling back to the out-of-root ext id path. Resolving just the parent keeps file symlinks from following onto their targets and merging sources, though directory symlinks inside the root get collapsed by the same resolution. New tests cover repo-relative paths, distinct same-basename files, and the incremental-merge prune case under a symlinked root.

Worth a look

  • Resolving the parent collapses in-root directory symlinks under a symlinked root — graphify/extract.py:10866 · Escalate · medium
    • NOT verified (no proof, no reproducing execution) — needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 3158 functions depend on the 332 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: extract() — 989 callers, 59 callees
  • new: test_pruning_a_deleted_page_keeps_the_reextracted_one() — 0 callers, 6 callees

Verification — 3158 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 2913 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

203 of 437 test file(s) selected (46%) via static blast radius.

  • tests/test_affected_overrides.py — impact
  • tests/test_astro_extraction.py — impact
  • tests/test_astro_import_ids.py — impact
  • tests/test_blade_extractor.py — impact
  • tests/test_build.py — impact
  • tests/test_builtin_global_type_refs.py — impact
  • tests/test_c_include_resolution.py — impact
  • tests/test_cache.py — impact
  • tests/test_case_sensitive_resolution.py — impact
  • tests/test_cjs_module_extension.py — impact
  • tests/test_cobol_extractor.py — impact
  • tests/test_cpp_method_declarations.py — impact
  • tests/test_cpp_nested_and_cli.py — impact
  • tests/test_cpp_objc_cross_file_calls.py — impact
  • tests/test_cpp_template_type_params.py — impact
  • tests/test_cross_extension_reexport_self_cycle.py — impact
  • tests/test_cross_language_call_resolution.py — impact
  • tests/test_cross_repo_external_call_guards.py — impact
  • tests/test_cross_repo_member_calls.py — impact
  • tests/test_csharp_call_site_generic_args.py — impact
  • tests/test_csharp_delegate_declarations.py — impact
  • tests/test_csharp_enum_members.py — impact
  • tests/test_csharp_field_generic_args.py — impact
  • tests/test_csharp_foreach_loop_variable.py — impact
  • tests/test_csharp_generic_callsites.py — impact
  • tests/test_csharp_interface_dispatch.py — impact
  • tests/test_csharp_member_calls.py — impact
  • tests/test_csharp_member_chain_calls.py — impact
  • tests/test_csharp_member_nodes.py — impact
  • tests/test_csharp_object_creation.py — impact
  • tests/test_csharp_partial_classes.py — impact
  • tests/test_csharp_tuple_type_refs.py — impact
  • tests/test_csharp_type_resolution.py — impact
  • tests/test_csharp_underscore_collisions.py — impact
  • tests/test_csharp_var_call_return_type.py — impact
  • tests/test_csharp_verbatim_interpolation.py — impact
  • tests/test_definition_file_portability.py — impact
  • tests/test_detect.py — impact
  • tests/test_dotnet.py — impact
  • tests/test_duplicate_annotation_edges.py — impact
  • tests/test_ejs_extractor.py — impact
  • tests/test_elixir_defdelegate.py — impact
  • tests/test_elixir_import_resolution.py — impact
  • tests/test_elixir_keyword_def_calls.py — impact
  • tests/test_elixir_qualified_calls.py — impact
  • tests/test_elixir_unqualified_call_scope.py — impact
  • tests/test_erlang_extractor.py — impact
  • tests/test_ext_tagged_id_clash.py — impact
  • tests/test_extract.py — impact
  • tests/test_extract_cache_location.py — impact
  • … and 153 more

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

No difference found (not proven): No behavior difference found in extract (not a proof).

The verifier ran both versions of extract on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

· 1 grounded finding(s) anchored inline below; 1 more finding(s) on lines outside this diff (see the check run).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — test_pruning_a_deleted_page_keeps_the_reextracted_one()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

extract(): source_file collapses to basename when the scan root is reached through a symlink (macOS /var, symlinked checkouts)

1 participant