{{ message }}
Repository navigation
No any speed increase #71
Description
I'm using an A100 running Qwen Image Edit 2511 fp8mixed, and its inference speed is approximately 1.01s/it
Afterwards, I saw this extension and tried using it to create OTFs from the bf16 version, or manually creating convolutional int8 safetensors.
However, I didn't get any speedup from it, even when using compile; the speed remained unchanged compared to fp8mixed.
Activity
BobJohnson24 commentedon Jun 7, 2026
That is very curious as the A100 should not have any hardware accelerated FP8. I don't really have any A100's lying around to test though. On all hardware that I have available there is a decent speed gain with INT8.
I guess it's possible our triton kernels are not properly working on an A100, but even then there should be at least some noticeable improvement. Seeing no speed-up at all is indeed highly suspect.
Airo0 commentedon Jun 7, 2026
I even tried replacing the silver's comfy_kitchen wheel and add
--enable-triton-backendto force the Triton backend to be enabled, but still nothing changed.Airo0 commentedon Jun 7, 2026
huanchen-stack/sglang@b2b01ba
I found the reason.
BobJohnson24 commentedon Jun 7, 2026
I am sorry to hear that and will keep this in mind if A100s ever go cheap.