Repository navigation
tests: add lora tester mixin for modular and migrate minimax h3. - #14605
Conversation
There was a problem hiding this comment.
I don't like the pipeline-specific condition here but I think it's a fair compromise.
There was a problem hiding this comment.
you can also check the inputs from pipe.blocks.inputs, you do need to build the pipeline first though
There was a problem hiding this comment.
I preferred that solution for a bit more optimization (i.e., skipping just after inspecting if the inputs don't exist) without having to build the pipeline. Is that okay for this test?
| pass | ||
|
|
||
|
|
||
| class TestMiniMaxH3ModularPipelineLoRA(MiniMaxH3ModularPipelineTesterConfig, ModularLoraTesterMixin): |
There was a problem hiding this comment.
The extra methods are specific to the transformer and transformer_ref related voodoo going on in the pipeline.
| class ModularLoraMemoryTesterMixin(BaseModularPipelineOutputMixin, LoraMemoryTesterMixin): | ||
| """LoRA x offloading tests for modular pipelines: group offloading composed with `load_lora_weights`.""" | ||
|
|
||
| @pytest.mark.skip( |
There was a problem hiding this comment.
We can add testing for this later.
| "transformer_ref": ["to_q", "to_k", "to_v", "to_out.0"], | ||
| } | ||
|
|
||
| def get_dummy_diffusers_lora_state_dict(self, transformer, prefix="transformer", rank=8, adaln_rank=2): |
There was a problem hiding this comment.
In the process, I also optimized the runtime of these tests. Previously, we were building the entire pipeline just to get the transformer from it. We already have all the components available when we're calling this method. So, this method now simply reuses them unlike:
diffusers/tests/lora/test_lora_layers_minimax_h3.py
Lines 104 to 113 in ee9f497
yiyixuxu
left a comment
yiyixuxu
left a comment
There was a problem hiding this comment.
thanks! I left some comments!
| denoiser components are not named `transformer`. | ||
| """ | ||
|
|
||
| @pytest.mark.skip( |
There was a problem hiding this comment.
it does save the weights though, for these components that are loaded at the time you run save_pretrained()
There was a problem hiding this comment.
I decided to delete the skip altogether because it can be tested with some additions to the base test suite. But so far our modular pipelines supporting LoRAs don't have this feature to be tested. So, when we do add something along these lines, we can revisit it. WDYT?
There was a problem hiding this comment.
you can also check the inputs from pipe.blocks.inputs, you do need to build the pipeline first though
2 participants
With more and more pipelines being released with Modular and supporting LoRA, it's important for us to ensure we test them as good as we do for standard pipelines that support LoRAs.
This PR, hence, adds a LoRA tester mixin class to our testing suite for Modular Pipelines. The mixin overrides
tests/pipelines/testing_utils/lora.pyand migrates the existing MiniMax H3 LoRA testing suite with it (as the existing one already makes use of Modular).We can add a LoRA tester class for the remaining Modular Pipelines that support it in a follow-up PR.