fix: language files are never parsed where file(1) is unavailable by ppcvote · Pull Request #5872 · ossf/cve-bin-tool · GitHub
Skip to content

fix: language files are never parsed where file(1) is unavailable - #5872

Merged
ffontaine merged 1 commit into
ossf:mainfrom
ppcvote:fix/windows-language-file-detection
Sep 29, 2026
Merged

ffontaine merged 1 commit into
ossf:mainfrom
ppcvote:fix/windows-language-file-detection

Conversation

@ppcvote

@ppcvote ppcvote commented Aug 9, 2026

Copy link
Copy Markdown

Fixes #5066.

The jar is a symptom. The defect is in checker selection in scan_file:

valid_file = valid_file | (file in output) if output else (file == filename_basename)

The conditional binds the whole right-hand side, so the branches are not "accumulate one way or the other" — they are accumulate, or assign. With output empty the loop overwrote valid_file every iteration, and only the last checker name in the dict could ever match.

output is filled by is_linux_kernel() only when file is on PATH, and file is a Unix utility.

Scale

Measured against the real valid_files dict rather than reasoned about — on a host without file, 17 of 18 language file types are never parsed:

conan.lock  pubspec.lock  .env  go.mod  pom.xml  package-lock.json  yarn.lock
.control  .pyd  cpanfile  composer.lock  requirements.txt  PKG-INFO:  METADATA:
renv.lock  Gemfile.lock  Cargo.lock          -> all dead
Package.resolved                              -> the only survivor, by sorting last

So this is not only jars: Java, Go, npm, yarn, Python, Rust, Ruby, PHP, Dart, Perl, R, conan and opkg all produce empty SBOM components on such a host, and the scan exits 0. For a vulnerability scanner, reporting clean because it never looked is the expensive part of the bug.

Why the Windows CI job never caught this

windows-latest carries Git for Windows, which ships file.exe in usr\bin. inpath("file") is True on the runner, CI takes the same branch Linux does, and the defect cannot appear there.

Confirmed on this machine — where file resolves to C:\Program Files\Git\usr\bin\file.exe, and the bug only shows once that directory is off PATH. That is also why @smirgel saw it on a plain Windows Server box and @terriko could not reproduce it.

Verification

Windows 10, driving the real VersionScanner against test/language_data/pom.xml:

file on PATH parser reached
before, CI-like env True True (masked)
before, reporter's env False False
after, CI-like env True True
after, reporter's env False True

Two dead ends in the thread, since they cost fifteen months

The missing Done scanning file is a red herring. That line is logged at the end of run_checkers, which scan_file only calls for executables and kernel images. It never appears for a language file, on any platform. @smirgel's observation was accurate but does not indicate the failure.

Line endings are not involved. The pom.xml inside log4j-core-2.24.3.jar is pure ASCII with LF endings, and validate_pom() returns True for it on a cp950 Windows host. I checked this first, since it was the standing hypothesis.

The change

The loop only ever accumulated, so it is now the any() it always was. That removes the precedence trap rather than re-parenthesising around it. The minimal alternative is one pair of brackets:

valid_file = valid_file | ((file in output) if output else (file == filename_basename))

Happy to switch if you prefer the smaller diff.

On the test

It patches inpath to reproduce a host without file, so it exercises the defect on Linux CI rather than needing a Windows runner without Git.

I could not execute it locally, and would rather say so than imply otherwise: TestLanguageScanner.setup_class calls get_cvelist_if_stale(), so the entire class errors at setup without the full CVE database. The behaviour it asserts is exactly what the table above measures directly.

Worth considering separately: _validate_xml and this selection both fail closed and silently, and a scan that parses nothing is indistinguishable from a scan that found nothing. A single warning when a file matched PARSER_MATCH_FILENAMES but no parser ran would have surfaced this in a day rather than fifteen months. Happy to send that as its own PR if it is wanted.

Fixes ossf#5066.

The checker selection in scan_file read:

    valid_file = valid_file | (file in output) if output else (file == basename)

The conditional binds the whole right-hand side, so the two branches are not
"accumulate one way or the other" — they are accumulate, or ASSIGN. With
`output` empty the loop overwrote `valid_file` on every iteration and only the
last checker name in the dict could ever match.

`output` is filled by is_linux_kernel() only when `file` is on PATH, and `file`
is a Unix utility. So on a Windows host without it, 17 of the 18 language file
types are silently never parsed — `Package.resolved` survives alone, by sorting
last. Measured against the real `valid_files` dict:

    conan.lock pubspec.lock .env go.mod pom.xml package-lock.json yarn.lock
    .control .pyd cpanfile composer.lock requirements.txt PKG-INFO: METADATA:
    renv.lock Gemfile.lock Cargo.lock        -> all dead
    Package.resolved                          -> the only survivor

The reported symptom, a jar producing an SBOM with no Java component and a
scan exiting 0, is one instance. Java, Go, npm, yarn, Python, Rust, Ruby, PHP,
Dart, Perl, R, conan and opkg are all affected the same way, and a vulnerability
scanner reporting clean because it never looked is the expensive part.

WHY THE WINDOWS CI JOB DID NOT CATCH IT. `windows-latest` carries Git for
Windows, which ships `file.exe` in `usr\bin`. `inpath("file")` is therefore
True on the runner, CI takes the same branch Linux does, and the defect is
invisible there. Confirmed on this machine: `where file` resolves to
`C:\Program Files\Git\usr\bin\file.exe`, and the bug only appears once that
directory is removed from PATH.

Verified end to end on Windows 10, driving the real VersionScanner against
test/language_data/pom.xml:

                              file on PATH   parser reached
    before, CI-like env           True           True     (masked)
    before, reporter's env        False          False    <- the bug
    after,  CI-like env           True           True
    after,  reporter's env        False          True

The loop only accumulated, so it is now the `any()` it always was, which also
removes the precedence trap rather than re-parenthesising around it. The
minimal alternative is one pair of brackets; happy to switch if preferred.

Two notes on the issue thread, since they cost fifteen months:

- The missing "Done scanning file" is a red herring. That line is logged at the
  end of run_checkers, which scan_file only calls for executables and kernel
  images, so it never appears for a language file on any platform.
- Line endings are not involved. The pom.xml inside log4j-core-2.24.3.jar is
  pure ASCII with LF, and validate_pom() returns True for it on a cp950 host.

The added test patches `inpath` to reproduce a host without `file`, so it
exercises the defect on Linux CI too. I could not execute it locally: this test
class's setup_class calls get_cvelist_if_stale(), which wants the full CVE
database, so the whole class errors at setup here. The behaviour it asserts is
the one measured in the table above.

Signed-off-by: ppcvote <risky9763@gmail.com>
@ppcvote
ppcvote force-pushed the fix/windows-language-file-detection branch from 64340d3 to 8dce233 Compare August 10, 2026 07:07
@ffontaine

Copy link
Copy Markdown
Collaborator

@ffontaine
ffontaine merged commit ce321ff into ossf:main Sep 29, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: scan on windows not identifying jar file

2 participants