docs,t: expand checkout and pull sparse cone tests · git-lfs/git-lfs@e9c1bc2 · GitHub
Skip to content

Commit e9c1bc2

Browse files
chrisd8088chrisd8088
committed
docs,t: expand checkout and pull sparse cone tests
In commit 5aa7be5 of PR #5796 we added tests of the sparse checkout support provided by our "git lfs checkout" and "git lfs pull" commands, which makes use of the "git ls-files" command and the --sparse option that was introduced for that command in Git v2.35.0. In practice, the "git lfs checkout" and "git lfs pull" commands require Git v2.42.0 or higher to be available before they invoke "git ls-files", and otherwise fall back to using the "git ls-tree" command. We require at least Git v2.42.0 because that version introduced support for the "objecttype" field name in the "git ls-files" command's --format option and we depend on that field to be able to mimic the output format of the "git ls-tree" command with the "git ls-files" command. We noted these details in commit beae114 of PR #5699, when we revised the runScanLFSFiles() function in our "lfs" package to choose between the use of "git ls-files" and "git ls-tree". One difference between the "git ls-files" and "git ls-tree" commands, however, is that the former lists the files in the Git index (since we always pass the --cached option) while the latter lists the files in the Git tree associated with a given reference, which in the case of our "git lfs checkout" and "git lfs pull" commands is always the current "HEAD" symbolic reference. As a consequence, as discussed in issue #6004, if certain files are absent from the current working tree and Git index as the result of a partial clone or sparse checkout, the behaviour of the "git lfs checkout" and "git lfs pull" commands varies depending on the installed version of Git. If Git v2.42.0 or higher is installed, the "git lfs checkout" and "git lfs pull" commands invoke the "git ls-files" command and provide an "attr:filter=lfs" pathspec so the Git command will filter out files which do not match a Git LFS filter attribute. However, in order to be reported, Git LFS pointer files must exist in the Git index; if they only appear in the working tree or the Git tree associated with the "HEAD" reference, they will be ignored. (Note that in a non-bare repository, the "git ls-files" command will only match the "attr:filter=lfs" pathspec against attributes defined in ".gitattributes" files in the index or working tree, plus any local files such as the "$GIT_DIR/info/attributes" file. Any ".gitattributes" files that are present only in the Git tree associated with the "HEAD" reference will not be consulted. In a bare repository, meanwhile, the "git ls-files" command will by default not match the pathspec against attributes defined in ".gitattributes" files at all, regardless of whether such files exist in the index or in the tree referenced by "HEAD".) If a version of Git older than v2.42.0 is installed and so the "git ls-tree" command is invoked instead of the "git ls-files" command, then a full list of the files in the tree-ish referenced by "HEAD" is returned. The "git lfs checkout" and "git lfs pull" commands will then attempt to check out the Git LFS objects associated with all the Git LFS pointer files found in this list. In the case of the "git lfs pull" command, it will also try to fetch those objects if they are not already present in the local storage directories. (Note, though, that when the "git lfs checkout" and "git lfs pull" commands retrieve a list of files using the "git ls-tree" command, they do not check whether the pointer files they find in that list actually match Git LFS filter attributes in any ".gitattributes" or other Git attributes files. So a user may remove all the ".gitattributes" files from their working tree and index, commit those changes to "HEAD", and the Git LFS commands will still attempt to check out objects for any files found in the "HEAD" commit's tree that can be parsed as valid pointers. When the "git ls-files" command is used instead of the "git ls-tree" command to retrieve a file list, this legacy behaviour does not occur, because the "attr:filter=lfs" pathspec requires that the "git ls-files" command only return a list of files which match at least one Git LFS filter attribute.) In subsequent commits we will alter how the "git lfs checkout" and "git lfs pull" commands operate within bare repositories and how they handle file paths, including by changing the current working directory to the root of the current working tree, if one is present. Of necessity, our tests and documentation will also be expanded to reflect the variable behaviour of the "git lfs pull" command in particular, since its effects in a bare repository depend in part on the installed version of Git. Before we make these changes, we first revise our existing tests of the "git lfs checkout" and "git lfs pull" commands with partial clones and sparse checkouts so that the tests confirm the key differences in behaviour when the installed version of Git is v2.42.0 or higher. Our tests now demonstrate that with an older version of Git, objects will be fetched (in the case of the "git lfs pull" command) and checked out for all Git LFS files, including those outside the configured sparse cone. We also update the manual pages for these commands to include an explanation of how their operation varies depending on the installed version of Git, how this may affect repositories with partial clones and sparse checkouts, and the options available to users if they find the "git lfs checkout" and "git lfs pull" commands appear to be ignoring certain files. As well, we edit the initial section in our git-lfs-pull(1) manual page where we incorrectly state that the command is always equivalent to running "git lfs fetch" followed by "git lfs checkout", and fix the formatting of the example commands provided in this section. When we converted our manual page source files from the Ronn format to AsciiDoc in commit 0c66dcf of PR #5054, the two example commands in this section were accidentally merged onto a single line, and the "<remote>" option for the "git lfs fetch" command was elided. We therefore restore the original version of these two example commands and add leading shell prompt indicators to further clarify that the example includes two separate commands.
1 parent 5d2645d commit e9c1bc2

4 files changed

Lines changed: 107 additions & 24 deletions

File tree

‎docs/man/git-lfs-checkout.adoc‎

Lines changed: 17 additions & 1 deletion

‎docs/man/git-lfs-pull.adoc‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,28 @@ git-lfs-pull - Download all Git LFS files for current ref & checkout
1313
Download Git LFS objects for the currently checked out ref, and update
1414
the working copy with the downloaded content if required.
1515

16-
This is equivalent to running the following 2 commands:
17-
18-
git lfs fetch [options] [] git lfs checkout
16+
This is generally equivalent to running the following two commands:
17+
18+
....
19+
$ git lfs fetch [options] [<remote>]
20+
$ git lfs checkout
21+
....
22+
23+
If the installed Git version is at least 2.42.0, in a non-bare repository
24+
this command will by default fetch and check out Git LFS objects for files
25+
only if they are present in the Git index and if they match a Git LFS
26+
filter attribute from a `.gitattributes` file that is present in either
27+
the index or the current working tree (or, as is always the case, if
28+
they match a Git LFS filter attribute in a local `gitattributes` file
29+
such as `$GIT_DIR/info/attributes`). These constraints do not apply
30+
with prior versions of Git.
31+
32+
In a repository with a partial clone or sparse checkout, it is therefore
33+
advisable to check out all `.gitattributes` files from `HEAD` before
34+
using this command, if Git v2.42.0 or later is installed. Alternatively,
35+
the `GIT_ATTR_SOURCE` environment variable may be set to `HEAD`, which
36+
will cause Git to only read attributes from `.gitattributes` files in
37+
`HEAD` and ignore those in the index or working tree.
1938

2039
== OPTIONS
2140

@@ -52,6 +71,6 @@ you're tracking first, or origin otherwise.
5271

5372
== SEE ALSO
5473

55-
git-lfs-fetch(1), git-lfs-checkout(1), gitignore(5).
74+
git-lfs-fetch(1), git-lfs-checkout(1), gitattributes(5), gitignore(5).
5675

5776
Part of the git-lfs(1) suite.

‎t/t-checkout.sh‎

Lines changed: 37 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -686,10 +686,11 @@ begin_test "checkout: sparse with partial clone and sparse index"
686686
(
687687
set -e
688688

689-
# Only test with Git version 2.42.0 as it introduced support for the
690-
# "objecttype" format option to the "git ls-files" command, which our
691-
# code requires.
692-
ensure_git_version_isnt "$VERSION_LOWER" "2.42.0"
689+
# Only test with Git version 2.25.0 as it introduced the
690+
# "git sparse-checkout" command. (Note that this test also requires
691+
# that the "git rev-list" command support the "tree:0" filter, which
692+
# was introduced with Git version 2.20.0.)
693+
ensure_git_version_isnt "$VERSION_LOWER" "2.25.0"
693694

694695
reponame="checkout-sparse"
695696
setup_remote_repo "$reponame"
@@ -736,30 +737,55 @@ begin_test "checkout: sparse with partial clone and sparse index"
736737
assert_local_object "$contents2_oid" 1
737738
refute_local_object "$contents3_oid"
738739

739-
# Git LFS objects associated with files outside of the sparse cone
740-
# should be ignored entirely, rather than just skipped.
741740
git lfs checkout 2>&1 | tee checkout.log
742741
if [ "0" -ne "${PIPESTATUS[0]}" ]; then
743742
echo >&2 "fatal: expected checkout to succeed ..."
744743
exit 1
745744
fi
746-
grep -q 'Skipped checkout for "out-dir/c.dat"' checkout.log && exit 1
745+
746+
# When Git version 2.42.0 or higher is available, the "git lfs checkout"
747+
# command will use the "git ls-files" command rather than the
748+
# "git ls-tree" command to list files. Git v2.42.0 introduced support
749+
# in the "git ls-files" command for the "objecttype" format option and
750+
# so Git LFS can use this command to avoid checking out objects outside
751+
# the sparse cone. Otherwise, all Git LFS objects will be checked out.
752+
gitversion="$(git version | cut -d" " -f3)"
753+
set +e
754+
compare_version "$gitversion" '2.42.0'
755+
result=$?
756+
set -e
757+
if [ "$result" -eq "$VERSION_LOWER" ]; then
758+
grep 'Skipped checkout for "out-dir/c.dat"' checkout.log
759+
760+
[ -f "out-dir/c.dat" ]
761+
[ "$(pointer $contents3_oid 1)" = "$(cat "out-dir/c.dat")" ]
762+
else
763+
grep -q 'Skipped checkout for "out-dir/c.dat"' checkout.log && exit 1
764+
765+
[ ! -e "out-dir/c.dat" ]
766+
fi
747767

748768
# Fetch all Git LFS objects, including those outside the sparse cone.
749769
git lfs fetch origin main
750770

751771
assert_local_object "$contents3_oid" 1
752772

753-
# Git LFS objects associated with files outside of the sparse cone
754-
# should not be checked out.
755773
git lfs checkout 2>&1 | tee checkout.log
756774
if [ "0" -ne "${PIPESTATUS[0]}" ]; then
757775
echo >&2 "fatal: expected checkout to succeed ..."
758776
exit 1
759777
fi
760-
grep -q 'Checking out LFS objects: 100% (3/3), 3 B' checkout.log && exit 1
761778

762-
[ ! -e "out-dir/c.dat" ]
779+
if [ "$result" -eq "$VERSION_LOWER" ]; then
780+
grep 'Checking out LFS objects: 100% (3/3), 3 B' checkout.log
781+
782+
[ -f "out-dir/c.dat" ]
783+
[ "$contents3" = "$(cat "out-dir/c.dat")" ]
784+
else
785+
grep -q 'Checking out LFS objects: 100% (3/3), 3 B' checkout.log && exit 1
786+
787+
[ ! -e "out-dir/c.dat" ]
788+
fi
763789
)
764790
end_test
765791

‎t/t-pull.sh‎

Lines changed: 30 additions & 8 deletions

0 commit comments

Comments
 (0)