load_in_16bit = True on a fine-grained fp8 checkpoint is meant to load 16-bit weights. Since #11531, FastModel passes dequantize = True to transformers' fp8 quantizer for this. The FastLanguageModel fast text path (Llama, Qwen, Mistral, ... through dispatch_model.from_pretrained) never does. On GPUs with native FP8 (sm89+: L4, H100, B200), transformers then keeps every projection as an FP8Linear with fp8 weights, even though 16-bit was asked for.

Reproduced on B200 with transformers 5.17.0, a tiny block-fp8 Llama (128 x 128 scales), load_in_16bit = True:

Saving such a model was also broken, on both loaders. With dequantize = True, transformers 5.x keeps the Fp8Dequantize load conversions on the model (model._weight_conversions), and save_pretrained reverses them into Fp8Quantize. On B200, transformers 5.17.0, after a 16-bit load followed by save_pretrained and a plain reload:

Change

• New _fp8_dequantize_quantization_config(model_config) in loader_utils.py. It returns the checkpoint's own FineGrainedFP8Config with dequantize = True, or None for anything other than quant_method: fp8 or when transformers has no dequantize flag.
• FastLanguageModel.from_pretrained sets it as the quantization_config, only when:
  • load_in_16bit is explicit;
  • no 4-bit / 8-bit is requested;
  • fast_inference is off;
  • the user passed no quantization_config.
• New _drop_fp8_dequantize_conversions(model), called after a 16-bit load in both loaders:
  • drops the generic weight + weight_scale_inv -> weight converter, whose only op is Fp8Dequantize;
  • a model converter that had Fp8Dequantize prepended (expert merges) is replaced by its quantizer-free form from get_model_conversion_mapping(model), so the experts still save per expert;
  • if no such form exists, it keeps the converter and says so;
  • it does nothing when no Fp8Dequantize conversion is present.
• Unchanged:
  • loads without the flag;
  • ModelOpt / compressed-tensors checkpoints;
  • what FastModel loads (only what it saves);
  • transformers versions without the dequantize flag.

Validation

New tests/test_fp8_16bit_text_loader_dequant.py:

• the helper returns dequantize = True for a block-fp8 config (dict or object), and None for no config, bitsandbytes or ModelOpt;
• an enum quant_method is read by its value;
• in a subprocess, FastLanguageModel on a tiny block-fp8 Llama: with load_in_16bit = True the model has 0 fp8 params and the quantizer got dequantize = True; without the flag the config is not touched;
• a dequantizing load of a tiny block-fp8 Llama and Qwen3-MoE, then _drop_fp8_dequantize_conversions, save_pretrained and a reload: only bf16 tensors are written, no quantization_config, and every weight reloads bit-identical.

With tests/test_fp8_leftover_dequant_16bit.py and tests/test_modelopt_fp8_loading.py on B200:

• transformers 5.17.0: 77 passed, 1 skipped.
• transformers 4.57.6: the new tests skip (no dequantize flag); 47 passed, 4 failed, the same 4 test_modelopt_fp8_loading plan tests that fail on main.

Not covered:

• transformers 4.57.x: FineGrainedFP8Config there has no dequantize parameter, so both loaders still keep FP8Linear on an FP8-capable GPU, and there are no Fp8Dequantize conversions to drop. This PR does not change that.
• Merged saves: save_pretrained_merged and GGUF export were not run on a dequantized fp8 model; only save_pretrained.
• GPUs without native FP8 (T4, A100): transformers already dequantizes there, so they were never affected.