Autominsize support by gnkow · Pull Request #6284 · git-lfs/git-lfs · GitHub
Skip to content

Autominsize support - #6284

Draft
gnkow wants to merge 7 commits into
git-lfs:mainfrom
gnkow:autominsize
Draft

gnkow wants to merge 7 commits into
git-lfs:mainfrom
gnkow:autominsize

Conversation

@gnkow

@gnkow gnkow commented Jun 17, 2026

Copy link
Copy Markdown

This PR adds a --min-size flag to git lfs track and fixes a build failure when no git tags are present.

The new --min-size flag automatically tracks files exceeding a given size threshold:

  • git lfs track --min-size 10MB sets lfs.autotracksize in .lfsconfig and adds * filter=lfs to .gitattributes
  • Existing files above the threshold are touched so they appear as modified on the next git add
  • Uses human-readable size units (KB, MB, GB, KiB, MiB, GiB)
  • Supports --dry-run to preview without making changes
  • The clean filter passes through files below the threshold unchanged

@chrisd8088 chrisd8088 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks very much for this draft PR, and for trying to tackle a long-standing issue!

I typically avoid reviewing on PRs in a draft state, as I assume they are still a work in progress, but from your comment I think you are hoping for some discussion, so I thought I would at least make some initial comments.

As I understand it, the key design principle of this PR is to pass all files through Git LFS; that's the effect of a .gitattributes pattern which matches *.

This idea has been considered before; I think our primary concern is going to be the performance cost of running every file through the Git LFS filters. We'd definitely want to see some comparative timing measurements done on each of the platforms we support (Linux, macOS, and Windows) using a reasonably large Git repository as test data.

Git LFS was originally developed several large Git hosting services (Atlassian, GitHub, GitLab) as a kind of workaround which allowed them to avoid storing large binary files in their own platforms. By using .gitattributes patterns to invoke the Git LFS filters, though, they could limit the impact of this additional program and system to just a specific set of files.

As a high-level concern, if we were to now start recommending that users process every file through Git LFS, that seems like we are trending away from direction I think we likely would prefer, where Git itself can do more of the work necessary to store and transfer large files efficiently.

A few other notes I made while thinking about this approach:

  • If the minimum size is relatively small, files such as .gitattributes and .gitignore may be converted to Git LFS objects, which is not appropriate as Git expects these to have a specific format that it can read. All such Git special files would need to be excluded automatically.
  • Since files managed with Git LFS can be quite large, using os.ReadAll() may consume a considerable amount of memory. It would likely be necessary to spool the incoming data to a temporary file; our tools.Spool() function does this already and is used in our smudge filter code to avoid deadlocking the pipe with Git (see PR #1932).
  • If a file below the threshold is removed from the working tree and then git checkout is run to recreate it, Git will invoke Git LFS as a smudge filter, which assumes the file should contain a pointer and will report it as malformed when it does not. So the smudge filter would also need to be aware of the file size threshold and ignore non-pointers below that limit.
  • I happen to have spent the last week working to deal with the ramifications of the bug fix proposed in PR #6011, and so I know that once it's merged, our clean() function will rarely know the size of a file in advance. Specifically, the fileSize parameter is passed as -1 during clean filter operations because Git doesn't provide the size of the data it's sending, and although we currently try to work around this by checking the size of the corresponding file in the working tree, this introduces a race condition because the file might change after Git starts streaming it to our filter. Ultimately, we have to remove this check and so only by reading the data arriving from Git would be know if its size exceeds a given threshold.
  • Specifying the threshold value in our .lfsconfig file is, I think, going to be a bit fragile, because it means the overall mechanism depends on both a global-match pattern in .gitattributes and a Git configuration option. Instead, if we were going to try to implement a threshold like this at all, I would want to see if we could specify it directly in the .gitattributes entry as a custom key/value attribute along with filter=lfs. Of course, this does mean Git LFS would have to read and parse the .gitattributes file to learn what the current threshold is, if any.

Those are just some quick initial thoughts. Whether or not we pursue the approach taken in this PR, it's definitely worth considering the implications of such a design, so thank you very much for opening the PR and starting the discussion!

@gnkow

gnkow commented Jun 29, 2026

Copy link
Copy Markdown
Author

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants