You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c4f9ad5
Browse filesBrowse the repository at this point in the historyBrowse files
The page's examples called `pipe.to("cuda")`, which cannot work: the
checkpoint is ~134GB in bfloat16 and no single card holds it. Both now
register their components in a `ComponentsManager` with auto offloading,
which is the recipe the page's own Memory section documents.
The `t2va` and `fl2va` snippets passed no `num_frames`. They build the
pipeline without a `workflow=`, so the un-pruned signature merges the
input to None — `MiniMaxH3Ref2VASetupStep` declares it required with no
default and wins the first declaration over the layout step's 124 — and
the layout step reads that None rather than its own default. Fixed
upstream by #14234, which is not in this base yet, so both calls pass
`num_frames` explicitly until it is.
Also: the loading section now separates picking a workflow from loading
one, a tip introduces `pipe.doc`, the conditioner note moves up beside
the rest of the checkpoint layout, and the generation-as-a-reference
example loads both partitions in one call instead of describing a second
call it no longer makes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/source/en/api/pipelines/minimax_h3.md
+31-21Lines changed: 31 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,7 +34,9 @@ MiniMax-H3 was released as two checkpoint partitions that share every component
34
34
35
35
Everything but the transformer, i.e. the video VAE, the audio VAE, the Qwen3-VL conditioner, its tokenizer and processor, and the two schedulers, is shared and stored once.
36
36
37
-
All three tasks are workflows of the one [`MiniMaxH3Blocks`], and the repository carries one `modular_model_index.json` naming every component with its own loading spec. To serve a single task, pass the workflow to `from_pretrained`: it keeps only that workflow's blocks, so the pipeline's signature (`pipe.blocks.doc`) documents exactly that task's inputs, only that task's components are declared, and `load_components` fetches exactly their subfolders — a `t2va` / `fl2va` pipeline never touches `transformer_ref/`, a `ref2va` one never touches `transformer/`.
37
+
The conditioner is a `Qwen3VLForConditionalGeneration`, and MiniMax-H3 reads the *unnormalized* hidden state after its 50th decoder layer rather than the last one, so the full released checkpoint is used with its language-model head unused.
38
+
39
+
All three tasks are workflows of the one [`MiniMaxH3Blocks`], and the repository carries one `modular_model_index.json` naming every component with its own loading spec. To serve a single task, pass the workflow to `from_pretrained`: it keeps only that workflow's blocks, so the pipeline's signature (`pipe.doc`) documents exactly that task's inputs, only that task's components are declared, and `load_components` fetches exactly their subfolders — a `t2va` / `fl2va` pipeline never touches `transformer_ref/`, a `ref2va` one never touches `transformer/`.
To keep every workflow available on one pipeline instead, load the full blockset — it picks the workflow per call from the inputs it is passed — and restrict the *loading* per workflow with `load_components(workflow=...)`:
49
+
> [!TIP]
50
+
> `pipe.doc` prints what the pipeline in front of you takes and returns — every input with its default, the components it expects and the outputs it produces. Pruned to one workflow it describes exactly that task, which is the quickest way to see what a request needs before making one.
51
+
52
+
To keep every workflow available on one pipeline instead, leave the `workflow` argument out: the pipeline then picks the workflow per call from the inputs it is passed.
The *loading* can still go one workflow at a time. This one call fetches `transformer/` and every shared component, which serves both `t2va` and `fl2va`:
51
59
52
-
# loads `transformer/` and every shared component — this one call serves `t2va` and `fl2va`
A plain `load_components()` with no `workflow=` pulls **both** 61.7GB transformer partitions. That is a deliberate choice when a [`ComponentsManager`] with auto offloading manages the memory: the weights live in host RAM, the manager moves onto the accelerator just what each step needs, and switching workflows takes no extra call — when the `ref2va` denoiser wants the device, the strategy offloads the smallest set of components that frees enough room, which tends to be the idle `transformer/` partition rather than the (slightly larger) conditioner. See [Memory](#memory) for the recipes.
57
-
58
-
The conditioner is a `Qwen3VLForConditionalGeneration`, and MiniMax-H3 reads the *unnormalized* hidden state after its 50th decoder layer rather than the last one, so the full released checkpoint is used with its language-model head unused.
64
+
A plain `load_components()` with no `workflow=` pulls **both** 61.7GB transformer partitions, which is what lets one pipeline serve all three workflows without another loading call. Pair it with a [`ComponentsManager`] and auto offloading: the weights live in host RAM and the manager moves onto the accelerator just what each step needs, so when the `ref2va` denoiser wants the device the strategy offloads whatever frees enough room. See [Memory](#memory) for the recipes.
59
65
60
66
## Two schedulers
61
67
@@ -162,26 +168,30 @@ Two 80 GB cards run full bfloat16 this way with nothing streaming; two 48 GB car
162
168
163
169
```py
164
170
import torch
165
-
from diffusers import ModularPipeline
171
+
from diffusers importComponentsManager, ModularPipeline
166
172
from diffusers.utils import load_image
167
173
from diffusers.utils.export_utils import encode_video
0 commit comments