MiniMax Music3 preview: fix BF16 conv corruption, repair converter, w… · unslothai/audio.cpp@b0ab088 · GitHub
Skip to content

Commit b0ab088

Browse files
JoeMattieclaude
authored andcommitted
MiniMax Music3 preview: fix BF16 conv corruption, repair converter, wire web UI (0xShug0#243)
* Guard MiniMax Music3 conv weights against BF16 storage BF16 conv kernels are unsupported by the CUDA conv path: the naive im2col lowering asserts on the kernel type and the fast path the flow transformer's 1x1 convolutions take silently corrupts, saturating the generated audio into full-scale noise. This reproduces with any package whose conv kernels are stored BF16 and with weight_type=bf16 over the published F32-conv packages. conv_safe_storage_type() falls back to F32 for the plain conv weight loads (flow preprocess/postprocess, condition encoder proj, vocoder dec_in_proj) whenever the effective storage would be BF16. The weight-norm vocoder convolutions fold at load and are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Repair MiniMax Music3 converter and emit config sidecars The converter as shipped failed before writing any output: the variant spec builder crashed on the spec's tensor-less gguf source, and the generated spec did not pass audiocpp_gguf source matching, so every component conversion aborted. Conversion now runs with --allow-missing-model-spec until the spec sources are aligned with the runtime's filename-based component loading. Freshly converted packages also failed to load because the runtime's required config/<component>.json sidecars were never emitted; they are now copied from the source snapshot. Conv kernels are pinned to F32 on disk through --keep-type rules for every target type, matching the runtime guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Wire MiniMax Music3 into the native web UI Adds the catalog entry, advanced parameter controls, and the entry-file rule mapping the multi-component package to its language_model GGUF. The session accepts the canonical duration_seconds request option as an alias for duration_sec, which is what the UI's duration field sends. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 858df2b commit b0ab088

11 files changed

Lines changed: 82 additions & 13 deletions

File tree

‎include/engine/community_models/minimax_music3/assets.h‎

Lines changed: 10 additions & 0 deletions

‎scripts/minimax_music3/convert_gguf.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,24 @@
3434
("vocoder", "vocoder/diffusion_pytorch_model.safetensors", "vocoder.gguf"),
3535
]
3636

37+
# Conv kernels must not be stored BF16: the CUDA conv lowering corrupts on BF16 kernel
38+
# types (and the runtime guards this at load). Keep them F32 on disk for every target
39+
# type.
40+
KEEP_TYPES = {
41+
"transformer": ["preprocess_conv*=f32", "postprocess_conv*=f32"],
42+
"condition_encoder": ["proj.weight=f32"],
43+
"vocoder": ["dec_in_proj*=f32"],
44+
}
45+
46+
# Config sidecars the runtime requires next to the component GGUFs.
47+
CONFIG_SIDECARS = {
48+
"language_model": "language_model/config.json",
49+
"rvq_depth_decoder": "rvq_depth_decoder/config.json",
50+
"condition_encoder": "condition_encoder/config.json",
51+
"transformer": "transformer/config.json",
52+
"vocoder": "vocoder/config.json",
53+
}
54+
3755

3856
def parse_args() -> argparse.Namespace:
3957
parser = argparse.ArgumentParser(description=__doc__)
@@ -117,6 +135,7 @@ def write_variant_spec(args: argparse.Namespace) -> tempfile.TemporaryDirectory[
117135

118136
output_by_name = {name: output_name(args, component) for name, _input, _base in COMPONENTS for component in [next(c for c in COMPONENTS if c[0] == name)]}
119137
gguf_source = next(source for source in spec["sources"] if source["format"] == "gguf")
138+
gguf_source.setdefault("tensors", {})
120139
gguf_source["tensors"]["language_model_weights"] = f"model:{output_by_name['language_model']}"
121140
gguf_source["tensors"]["depth_decoder_weights"] = f"model:{output_by_name['rvq_depth_decoder']}"
122141
gguf_source["tensors"]["condition_encoder_weights"] = f"model:{output_by_name['condition_encoder']}"
@@ -151,11 +170,22 @@ def run_conversion(args: argparse.Namespace, component: tuple[str, str, str]) ->
151170
"--model-spec",
152171
str(args.model_spec),
153172
"--no-sidecars",
173+
# The runtime opens component GGUFs by filename rather than through spec tensor
174+
# ids, and the spec's gguf source does not yet match audiocpp_gguf's
175+
# source-matching rules, so conversion proceeds without a matching package spec.
176+
"--allow-missing-model-spec",
154177
]
178+
for keep in KEEP_TYPES.get(name, []):
179+
cmd.extend(["--keep-type", keep])
155180
if args.overwrite:
156181
cmd.append("--overwrite")
157182
print("[convert]", name, "->", output_path, flush=True)
158183
subprocess.run(cmd, cwd=REPO_ROOT, check=True)
184+
sidecar = CONFIG_SIDECARS.get(name)
185+
if sidecar is not None:
186+
config_dir = Path(args.output) / "config"
187+
config_dir.mkdir(parents=True, exist_ok=True)
188+
shutil.copyfile(Path(args.source) / sidecar, config_dir / f"{name}.json")
159189

160190

161191
def main() -> None:

‎src/community_models/minimax_music3/assets.cpp‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,4 +147,19 @@ std::shared_ptr<const MiniMaxMusic3Assets> load_minimax_music3_assets(
147147
return assets;
148148
}
149149

150+
assets::TensorStorageType conv_safe_storage_type(
151+
const assets::TensorSource & source,
152+
const std::string & tensor_prefix,
153+
assets::TensorStorageType requested) {
154+
if (requested == assets::TensorStorageType::BF16) {
155+
return assets::TensorStorageType::F32;
156+
}
157+
if (requested == assets::TensorStorageType::Native &&
158+
assets::ggml_type_for_tensor_dtype(source.require_metadata(tensor_prefix + ".weight").dtype) ==
159+
GGML_TYPE_BF16) {
160+
return assets::TensorStorageType::F32;
161+
}
162+
return requested;
163+
}
164+
150165
} // namespace engine::models::minimax_music3

‎src/community_models/minimax_music3/condition_encoder.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ MiniMaxMusic3ConditionWeights load_condition_weights(
7575
*out.store,
7676
source,
7777
"proj",
78-
storage_type,
78+
conv_safe_storage_type(source, "proj", storage_type),
7979
config.out_dim,
8080
config.condition_hidden_dim,
8181
3,

‎src/community_models/minimax_music3/flow_transformer.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ MiniMaxMusic3FlowWeights load_flow_weights(
5656
*out.store,
5757
source,
5858
"preprocess_conv",
59-
storage_type,
59+
conv_safe_storage_type(source, "preprocess_conv", storage_type),
6060
concat_channels,
6161
concat_channels,
6262
1,
@@ -84,7 +84,7 @@ MiniMaxMusic3FlowWeights load_flow_weights(
8484
*out.store,
8585
source,
8686
"postprocess_conv",
87-
storage_type,
87+
conv_safe_storage_type(source, "postprocess_conv", storage_type),
8888
config.in_channels,
8989
config.in_channels,
9090
1,

‎src/community_models/minimax_music3/session.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,7 @@ MiniMaxMusic3Request MiniMaxMusic3Session::parse_request(const runtime::TaskRequ
236236
if (out.lyrics.empty()) {
237237
throw std::runtime_error("MiniMax Music 3 requires lyrics");
238238
}
239-
if (const auto value = runtime::parse_finite_float_option(request.options, {"duration_sec"})) {
239+
if (const auto value = runtime::parse_finite_float_option(request.options, {"duration_sec", "duration_seconds"})) {
240240
out.duration_sec = *value;
241241
}
242242
if (const auto value = runtime::parse_i64_option(request.options, {"num_inference_steps"})) {

‎src/community_models/minimax_music3/vocoder.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ Music3VocoderWeights load_vocoder_weights(
8686
*out.store,
8787
source,
8888
"dec_in_proj",
89-
storage_type,
89+
conv_safe_storage_type(source, "dec_in_proj", storage_type),
9090
config.decoder_input_dim,
9191
config.latent_channels / 2,
9292
1,

‎webui/configs/model_params.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,13 @@
123123
{"name": "timesignature", "type": "text", "label": "【曲谱】timesignature", "default": "", "placeholder": "如 4"}
124124
],
125125

126+
"minimax_music3": [
127+
{"name": "num_inference_steps", "type": "number", "label": "Flow steps per window", "default": 30, "minimum": 1, "maximum": 200, "step": 1, "precision": 0, "info": "Flow-matching Euler steps per 200-frame denoising window."},
128+
{"name": "guidance_scale", "type": "slider", "label": "Flow guidance scale", "default": 1.7, "minimum": 0.0, "maximum": 10.0, "step": 0.1},
129+
{"name": "ar_guidance_scale", "type": "slider", "label": "AR guidance scale", "default": 1.5, "minimum": 0.0, "maximum": 10.0, "step": 0.1, "info": "Classifier-free guidance of the semantic and residual code sampling."},
130+
{"name": "top_k", "type": "number", "label": "top_k", "default": 50, "minimum": 1, "maximum": 1024, "step": 1, "precision": 0}
131+
],
132+
126133
"minimax_h3": [
127134
{"name": "num_inference_steps", "type": "number", "label": "Denoising steps", "default": 12, "minimum": 1, "maximum": 50, "step": 1, "precision": 0, "info": "Twelve denoising steps provide a practical quality and performance balance."},
128135
{"name": "num_frames", "type": "number", "label": "Output frames", "default": 241, "minimum": 5, "maximum": 1441, "step": 4, "precision": 0, "info": "Approximately 24 frames per output second; 241 frames produces about 10 seconds of audio."},

‎webui/configs/models_catalog.json‎

Lines changed: 1 addition & 0 deletions

‎webui/native/dist/index.html‎

Lines changed: 8 additions & 8 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)