{{ message }}
Repository navigation
[pull] master from Comfy-Org:master - #1149
Merged
Merged
Conversation
* Speed up the startup prune's owned-prefix check
mark_contents_missing_outside_prefixes tested every live AssetContent row
against every owned prefix with Path.is_relative_to, which walks the path's
parents on each call: rows x prefixes x depth. At 9k rows and 150 prefixes
that was 9.5s (15.5s with deeper output paths).
path_prefix_matcher normalizes the prefixes once and checks each row with a
normcase'd, separator-bounded str.startswith over a tuple, keeping
is_relative_to's component bounds and platform case rules. The same case
now takes about 0.04s, flat in prefix count and depth.
* Move path_prefix_matcher next to the SQL containment predicate
path_utils needs the same check for per-file tagging, and scanner_changes
already imports path_utils, so the matcher moves to app.assets.helpers
beside sql_path_under_prefix.
* Speed up asset tag derivation during scans
get_backend_system_tags_from_path checked every scanned file against every
model base with Path.is_relative_to, which dominated the fast scan: 37s of
build_asset_specs for 100k output files. Using path_prefix_matcher, the
separator-bounded string check the startup prune already uses, brings that
to about 7s.
* Keep path_prefix_matcher's parity for paths starting with two slashes
abspath keeps exactly two leading slashes, and pathlib treats that '//' as an
anchor of its own, so Path('//server/f').is_relative_to('/') is False, but
the string check accepted it. Prefixes are now split once by whether their
anchor is '//', and a candidate is only checked against prefixes with the
same anchor. The per-row check is still a single str.startswith.
* Build tag-derivation prefix matchers once per folder config
get_backend_system_tags_from_path built a path_prefix_matcher for input,
output, temp and every model category on each call, so most of its cost was
normalizing the same prefixes again for every scanned file. cached_prefix_matcher
memoizes construction on the raw prefix tuple, which the callers pass as
absolute folder paths; a folder-config change is a new key.
100k files, 58 model bases: tag derivation 81.8 -> 27.7 us/file, and
build_asset_specs CPU 11.55 -> 6.40s.
* State cached_prefix_matcher's absolute-prefix precondition
The docstring presented absolute prefixes as a property of the callers. It
is a precondition the caller must meet: folder_paths stores what it is given,
and a relative prefix is resolved once, on first use, and then frozen.
---------
Co-authored-by: guill <jacob.e.segal@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )