You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Faulted modules are skipped by cleanup after partial dynamic-VRAM prefetch failure #17175
On master 872eb6a3a5497721f53ebe82c0da9a3ba3414c13, a failure in pin_modules() can leave successful VBAR faults pinned after cleanup_prefetch_queues(). The queue still contains the raw upcoming module, which cleanup skips. A nonresident module that fails preparation may also lack its _prefetch cleanup record.
A model-free CPU regression module is available here. To reproduce in an isolated upstream checkout at the above commit, copy that test file into tests-unit/comfy_test/ and run:
Result: 4 failed, 4 passed. In test_failed_prefetch_keeps_all_faulted_modules_reachable, two VBARs fault successfully and preparation of the second module raises. Terminal cleanup makes no vbar_unpin calls; expected behavior is one unpin per successfully faulted VBAR and no remaining _prefetch records. Other cases cover failure during the batch and failure in real pin_modules() after casting.
Registering the batch before pinning and publishing minimal batched cleanup state before preparation makes all 8 tests pass. The original exception still propagates.
These tests inject errors and mock device boundaries; no model files or GPU workload are needed. This reproduces Python cleanup behavior, not a particular driver crash or live-model memory-pressure threshold.
Activity