Repository navigation
Conversation
|
|
|
The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:
Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows. |
1 similar comment
|
The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:
Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows. |
williamwen42
left a comment
There was a problem hiding this comment.
Address these concerns discovered.
|
Thanks for the review @williamwen42! I've addressed both review comments in e988db5 and also updated the test_sourceless_* tests in test/dynamo/test_misc.py (which previously relied on A().method having cls_source=None when A was a sourced local class) so they construct a sourceless MappingProxy lookup instead. Could you please re-approve the CI workflow run when you get a chance? |
Done |
|
@syerva-g can you rebase |
e988db5 to
8004523
Compare
done |
guilhermeleobas
left a comment
There was a problem hiding this comment.
Thanks! Just one nit
|
Thanks for the reviews @guilhermeleobas and @rtimpe! Addressed all comments in 7465dda. |
|
Please address test failures. |
19781ab to
4c59c57
Compare
|
@williamwen42 Fixed the |
@williamwen42 tests pass now, the 3 tests failing seems unstable and not related to this change in particular |
4c59c57 to
fc1f8d7
Compare
|
Rebased onto latest @williamwen42 @guilhermeleobas @rtimpe — could you please approve the CI workflow run and take another look when you get a chance so we can clear the changes-requested status and merge? Thanks! |
williamwen42
left a comment
There was a problem hiding this comment.
AI found some more soundness concerns related to changing function attributes.
|
|
||
| def enter(self, tx: "InstructionTranslatorBase") -> VariableTracker: | ||
| source = None if self.source is None else AttrSource(self.source, "__enter__") | ||
| fn_source = self.cls_source and self.get_source_by_walking_mro(tx, "__enter__") |
There was a problem hiding this comment.
AI-generated review comment - Finding 1: Guard newly MRO-sourced functions
This new MRO-derived
fn_sourceis passed toUserFunctionVariabledirectly. That source letsbind_argsreconstruct closure cells, but it does not guard that the function at the source is still the one Dynamo traced. RebindingHelper.valuetherefore reuses the cached graph: the old return value is retained and the old mutation is replayed into the replacement function's closure. The same unguarded construction occurs inUserDefinedObjectVariable.call_method(including__init__) and both context-manager entry/exit paths.Existing-work / severity: PARTIALLY_TRACKED. #197860 and #198124 cover the ordinary
resolve_type_attrbound-method case, so that subcase is existing and its severity is lowered. This finding is retained for the untracked PR-specific delta: #198055 newly makes MRO-derived sources reachable incall_method/__init__and context-manager__enter__/__exit__without installing the function guard.Reproduction:
import torch from torch._dynamo.testing import CompileCounterWithBackend from torch.testing._internal.common_utils import run_tests, TestCase class TestMethodRebind(TestCase): def test_locally_instantiated_method_rebind(self): old_log = [] new_log = [] class Helper: def value(self): old_log.append("old") return 1 def new_value(self): new_log.append("new") return 2 def fn(x): return x + Helper().value() backend = CompileCounterWithBackend("eager") compiled = torch.compile(fn, backend=backend, fullgraph=True) out1 = compiled(torch.ones(1)) Helper.value = new_value out2 = compiled(torch.ones(1)) self.assertEqual(out1, torch.tensor([2.0])) self.assertEqual(out2, torch.tensor([3.0])) self.assertEqual(old_log, ["old"]) self.assertEqual(new_log, ["new"]) self.assertEqual(backend.frame_count, 2) if __name__ == "__main__": run_tests()Observed on the exact current base and head:
base 4eab5fba: out1=[2] out2=[2] old_log=[] new_log=[] frames=1 head fc1f8d74: out1=[2] out2=[2] old_log=['old'] new_log=['old'] frames=1 guarded fix: out1=[2] out2=[3] old_log=['old'] new_log=['new'] frames=2The head result is particularly problematic: it silently applies the traced function's mutation to the replacement function's closure. I also reproduced this for
__init__and__enter__rebinding.Suggested fix:
Use
UserFunctionVariable.create_with_sourcewhenever the function source came from the class MRO, retaining the existing fallback when no MRO source is available:# UserDefinedObjectVariable.call_method if source_fn is not None: fn_vt = variables.UserFunctionVariable.create_with_source(method, source_fn) else: fn_vt = variables.UserFunctionVariable( method, source=source and AttrSource(source, "__func__") ) return UserMethodVariable(fn_vt, self, source=source).call_function( tx, args, kwargs ) # UserDefinedObjectVariable.resolve_type_attr if var_source is not None: fn_vt = variables.UserFunctionVariable.create_with_source( type_attr, var_source ) else: fn_vt = variables.UserFunctionVariable(type_attr, source=fn_source) return variables.UserMethodVariable(fn_vt, self, source=source) # GenericContextWrappingVariable.enter/exit if fn_source is not None: fn_vt = variables.UserFunctionVariable.create_with_source(fn, fn_source) else: fn_vt = variables.UserFunctionVariable(fn, source=fn_source)
create_with_sourceinstalls the requiredCLOSURE_MATCHguard. PR #198124 covers the ordinaryresolve_type_attrsite when composed with this change, but notcall_methodor context-manager entry/exit, so those sites still need to be handled here or coordinated explicitly.Verification used source-spliced worktrees at exact base
4eab5fba(then-currentorigin/main, gap 0) and headfc1f8d74; the only environment adaptation was a compatibility-onlyHealthCheckHookimport shim. All five tests added by this PR still passed with the guarded patch.
| type_attr = unpatched_nn_module_init | ||
|
|
||
| can_use_mro_source = self.cls_source is not None and self.source is not None | ||
| can_use_mro_source = self.cls_source is not None |
There was a problem hiding this comment.
AI-generated review comment - Finding 2: Recover sources for
lru_cacheand__getattr__methodsWidening
can_use_mro_sourcefixes the plain-function branch, but two adjacent method paths still ignore it. Thelru_cachebranch passes the incoming instancesource, which isNonefor a locally instantiated object, andcall_getattr_fallbackderives__getattr__only fromself.source, which is alsoNone. Their closure cells therefore remain sourceless and their mutations are silently dropped even though both functions compile withfullgraph=True.Existing-work / severity: EXISTING-IN-SCOPE; lowered. Both reproductions fail identically on the base and head, so this is not a regression introduced by #198055. It is an incomplete-fix gap within #197896: that issue tracks dropped closure mutations broadly, but no live issue or PR separately covers the
lru_cacheand custom__getattr__variants.Reproduction (
lru_cache(0)disables caching so this only tests closure handling):import functools import torch from torch._dynamo.testing import CompileCounter from torch.testing._internal.common_utils import run_tests, TestCase class TestMissingMethodSources(TestCase): def test_lru_cache_method_closure_mutation(self): log = [] class Helper: @functools.lru_cache(0) def touch(self): log.append("touch") def fn(x): Helper().touch() return x + 1 counter = CompileCounter() compiled = torch.compile(fn, backend=counter, fullgraph=True) self.assertEqual(compiled(torch.ones(1)), torch.tensor([2.0])) self.assertEqual(compiled(torch.ones(1)), torch.tensor([2.0])) self.assertGreater(counter.frame_count, 0) self.assertEqual(log, ["touch", "touch"]) def test_getattr_closure_mutation(self): log = [] class Helper: def __getattr__(self, name): log.append(name) return 1 def fn(x): return x + Helper().missing counter = CompileCounter() compiled = torch.compile(fn, backend=counter, fullgraph=True) self.assertEqual(compiled(torch.ones(1)), torch.tensor([2.0])) self.assertEqual(compiled(torch.ones(1)), torch.tensor([2.0])) self.assertGreater(counter.frame_count, 0) self.assertEqual(log, ["missing", "missing"]) if __name__ == "__main__": run_tests()Exact base and head both produced:
GETATTR frames=1 log=[] LRU frames=1 log=[] FAILED (failures=2)Suggested fix:
Give the wrapper's underlying function the MRO source while preserving the bound-method source. For
__getattr__, keep the bound-method and function sources separate and usecreate_with_sourcefor the MRO-derived function so this does not introduce Finding 1's rebinding bug:elif is_lru_cache_wrapped_function(type_attr): fn_source = source if can_use_mro_source: fn_source = self.get_source_by_walking_mro(tx, name) fn_vt = variables.WrapperUserFunctionVariable( type_attr, "__wrapped__", source=fn_source ) return variables.WrapperUserMethodVariable(fn_vt, self, source=source)method_source = None if self.source: method_source = AttrSource(self.source, "__getattr__") if self.cls_source is not None: fn_source = self.get_source_by_walking_mro(tx, "__getattr__") fn_vt = variables.UserFunctionVariable.create_with_source( getattr_fn, fn_source ) else: fn_source = method_source and AttrSource(method_source, "__func__") fn_vt = variables.UserFunctionVariable(getattr_fn, source=fn_source) out = variables.UserMethodVariable( fn_vt, self, source=method_source ).call_function(tx, [variables.ConstantVariable.create(name)], {})With this patch, the same test produced:
GETATTR frames=2 log=['missing', 'missing'] LRU frames=2 log=['touch', 'touch'] OKVerification used source-spliced worktrees at exact base
4eab5fbaand headfc1f8d74; the only environment adaptation was the compatibility-only import shim noted in Finding 1. The PR's five new tests also passed with this patch.
| return x + 1 | ||
|
|
||
| with torch._dynamo.config.patch(enable_trace_load_build_class=True): | ||
| g_nonlocal = torch.compile(f_nonlocal, backend="eager") |
There was a problem hiding this comment.
AI-generated review comment - Finding 3: Separate redundant build-class coverage from missing supported-path coverage
There are two independent coverage concerns here:
test_build_class_in_compiled_fn_nonlocal_rebindexercises behavior already covered more precisely bytest_misc.py::MiscTests::test_build_class_closure_rebinding, which asserts that rebinding a materialized sourceless build-class cell remains unsupported underfullgraph=True. Because this new test omitsfullgraph=True, it succeeds through graph-break/eager fallback and does not exercise this PR's production change. It can be dropped as redundant.- Separately, the supported locally defined class path still lacks coverage for
nonlocalcell reassignment. The existing tests and Findings 1/2 cover container mutation, function-identity guarding, and thelru_cache/__getattr__branches; none coversSTORE_DEREFthrough an ordinary method whose class is defined outside the compiled function.The distinction is observable on the exact current base and head:
added build-class test: base PASS, head PASS added build-class test + fullgraph=True: base Write to immutable cell, head Write to immutable cell supported-path test below: base Write to immutable cell, head PASS (n == 2)Suggested fix:
Drop the newly added fallback-only build-class test because that unsupported path is already pinned elsewhere. Independently add the following supported-path regression test alongside the other new closure-mutation tests.
Suggested test:
def test_locally_defined_method_nonlocal_rebind(self): n = 0 class Helper: def touch(self): nonlocal n n += 1 def fn(x): Helper().touch() return x + 1 compiled = torch.compile(fn, backend="eager", fullgraph=True) self.assertEqual(compiled(torch.ones(1)), torch.tensor([2.0])) self.assertEqual(compiled(torch.ones(1)), torch.tensor([2.0])) self.assertEqual(n, 2)This is not a semantic replacement for the existing build-class test: one confirms the unsupported build-class boundary, while the new test fills missing coverage for the supported ordinary-method path.
|
Thanks for the great catch @williamwen42! You're completely right on both points:
Ready for another look whenever you have a chance! |
|
Please address test failures. |
ae4eca8 to
8c883e2
Compare
…cells (pytorch#197896) Fixes pytorch#197896. When a locally defined class is instantiated inside a compiled function (e.g. `Helper().touch()`), the instance itself is sourceless (`self.source is None`), but its class source (`self.cls_source`) is available. Use `self.cls_source` to resolve the method/property/__enter__/__exit__ source via MRO walking so that `UserFunctionVariable.bind_args` tracks closure cells with a valid `ClosureSource` and replays in-place mutations to captured containers.
… sources to lru_cache/__getattr__
…k xfail - call_getattr_fallback: inspect.getattr_static(type(value), '__getattr__') also finds a __getattr__ defined on the metaclass (EnumType.__getattr__ on Python < 3.12), which is not in type(value).__mro__. Walking the MRO for it raised Unsupported, which _load_attr's python-constant fast path turned into a real AttributeError -> InternalTorchDynamoError (test_compilation_evnum_hasattr_fail on 3.11). Only source/guard __getattr__ when a class in the MRO defines it. - test_nested_graph_breaks_wrapped: drop the expectedFailure for test_udf_classes_reconstruction_nested_graph_breaks -- the MRO-sourced CLOSURE_MATCH guard now makes it recompile correctly when the class is rebound, so it was reported as 'Unexpected success'. - test_functions: noqa B019 on the intentional lru_cache method and reflow an overlong assertEqual for PYFMT.
…recompile test test_default_guard_filter_keeps_the_pytree_registry_keys_match counts the guards that the default guard filter keeps over a control filter. With this PR, methods resolved through the class MRO of a locally constructed object are built with UserFunctionVariable.create_with_source, which installs a CLOSURE_MATCH guard (serialized as FUNCTION_CODE_MATCH). pytree.tree_flatten constructs a TreeSpec during tracing, so TreeSpec.__init__ and TreeSpec.__post_init__ now get code checks -- exactly the guards that make a rebinding of those methods recompile instead of silently reusing the graph. Update the expected FUNCTION_CODE_MATCH count from 4 to 6 and document which functions they cover.
The previous commit installed CLOSURE_MATCH on every MRO-sourced method, including methods of objects that already have a source. Upstream sources those through the class MRO too but does not guard them (pytorch#197860, which pytorch#198124 addresses), and precompile now relies on that: since pytorch#200823, test_package.py's unnamed-scope tests call an exec()-defined method of a module-level object, and the new guard on it cannot be made portable, so saving the package raised "CLOSURE_MATCH guard cannot be serialized". Keep the guard for the sources this PR introduces - methods of objects constructed during tracing, which are reachable only through cls_source - and give objects that have a source back exactly the function source they had before.
b8802ba to
4be1753
Compare

Fixes #197896
Summary
When a user-defined class or context manager is instantiated inside a
@torch.compileregion (self.source is None),UserDefinedObjectVariablestill tracks the class provenance viaself.cls_source. However:UserDefinedObjectVariable.resolve_type_attrrequiredself.cls_source is not None and self.source is not Noneforcan_use_mro_source, droppingsource_fnwhen binding methods on newly constructed instances (and similarly incall_methodandresolve_data_descriptor).GenericContextWrappingVariable.enter/exitdid not passsource_fnviaget_source_by_walking_mrowhenself.cls_sourcewas present.NestedUserFunctionVariable.get_functiondid not register materialized closure cells inside_effects.track_cell_existingwhenallow_sourced_cells=True(__build_class__).Because the unbound method lacked a
Source,UserFunctionVariable.bind_argsbuilt the method's closure cells without aSource, causingSideEffectsto treat closure-captured lists/dicts asMutationNewand silently drop mutations upon function exit.Test Plan
pytest test/dynamo/test_functions.py -k test_locally_defined_class_closure_mutationcc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @kadeng @chauhang @amjames @jataylo @azahed98