{{ message }}
Repository navigation
Conversation
`git_hashsig_create_fromfile` reads the file in 4096 byte blocks and calls `hashsig_add_hashes` once per block. That function started a fresh run at the beginning of every call and recorded whatever run was in progress when it ran out of data, so a run of characters that spanned a block boundary was hashed as two shorter runs instead of one. For content whose lines are shorter than a block this is invisible, because every run ends at a newline. For a file with lines longer than 4096 characters the block boundaries fall in the middle of runs, and the resulting signature no longer matches the one `git_hashsig_create` computes over the same bytes in memory. Rename detection compares an in-memory blob against an on-disk file, so identical content was scored as dissimilar and renames of such files were missed. Keep the in-progress run in `hashsig_in_progress` so it continues across calls, only record a run once it has actually ended, and flush the final run before finalizing. Fixes libgit2#7196
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What
git_hashsig_create_fromfilereads the file in 4096-byte blocks and callshashsig_add_hashesonce per block. That function started a fresh run at thebeginning of every call, and recorded whatever run was in progress when it ran
out of data. A run of characters spanning a block boundary was therefore hashed
as two shorter runs rather than one.
For content whose lines are shorter than a block this never shows, because every
run ends at a newline anyway. For a file with lines longer than 4096 characters
the block boundaries land in the middle of runs, so the signature no longer
matches the one
git_hashsig_createcomputes over the same bytes in memory.Rename detection compares an in-memory blob against an on-disk file, so
identical content scored as dissimilar and renames of such files were missed.
That is the behaviour reported in #7196.
The fix
Keep the in-progress run in
hashsig_in_progressso it continues across calls,record a run only once it has actually ended, and flush the trailing run before
finalizing. The in-memory path is unaffected: it makes a single call, so the
trailing run that used to be recorded at the end of the buffer is now recorded
by the flush instead.
Reproduction and evidence
Using the reporter's recipe (six files of random characters on a single line,
sizes 1024 to 32768, all renamed,
GIT_DIFF_FIND_RENAMES | GIT_DIFF_FIND_FOR_UNTRACKED,threshold 50):
Before — only the files at or under one block are matched, exactly as reported:
After:
Test
core::hashsig::similarity_metric_long_linesextends the invariant the existingsimilarity_metrictest already checks for short data — that a signature builtfrom a file equals one built from the same bytes in memory — to content that
spans several read blocks with no newlines. It uses deterministic content rather
than random data.
Without the change it fails with
100 != 68; with the change it passes.Testing performed
Built with CMake 4.3.3 / Ninja 1.13.2 / Apple clang 21, Debug, macOS arm64.
libgit2_tests— 3031 passed, 22 skipped, 0 failedutil_tests— 441 passed, 10 skipped, 0 failedcore::hashsig,diff(219),status(84),merge(181),rebase(53),checkout(168) — all greenNote this changes the signature computed for on-disk files with lines longer
than 4096 characters, which is the point of the fix; similarity scores for such
files move toward the in-memory result. No existing test changed behaviour.
Only tested on macOS arm64; I have not run the Windows or Linux CI matrix
locally.
AI assistance disclosure: this change — the diagnosis, the patch, the test, and
this description — was produced with AI assistance. The reproduction, the
red/green verification, and the test runs reported above were actually executed
and the numbers are real. No human has separately reviewed the change.