Problem Description

When no quantization is specified in an API request, Unsloth Studio automatically uses the model’s default quantization, which is UD-Q5_K_XL in my case. I would prefer it to default to Q8_0 instead.

Currently, there is no option to set a preferred default quantization for a specific model. This is inconvenient because if the wrong quantization is loaded, I have to either manually unload and reload the model or specify the correct quantization explicitly in the first request.

Current Behavior

• If no quantization is specified in the request, Unsloth Studio uses the model’s default quantization, which is UD-Q5_K_XL in my case.
• There is no option to choose a preferred default quantization for each model.
• If Q8_0 is required, the user must either manually unload and reload the model or specify it explicitly in the first request.

Proposed Solution

Add an option in the model menu to select a preferred default quantization format for a given model.

For example, allow the user to set Q8_0 as the preferred default quantization for a specific model. If no quantization is specified in an API request, Unsloth Studio should load the model using this selected default.

Expected Outcome

• Users can set their preferred default quantization for each model.
• Models will load automatically with the correct quantization when none is specified in the request.
• Manual unloading and reloading, or explicit quantization specification in the first request, will no longer be necessary.