docs: correct the enable_gqa / cuDNN explanation by yiyixuxu · Pull Request #14561 · huggingface/diffusers · GitHub
Skip to content

docs: correct the enable_gqa / cuDNN explanation - #14561

Merged
yiyixuxu merged 2 commits into
mainfrom
gqa-cudnn-docs
Aug 26, 2026
Merged

yiyixuxu merged 2 commits into
mainfrom
gqa-cudnn-docs

Conversation

@yiyixuxu

Copy link
Copy Markdown
Collaborator

No description provided.

The Krea 2 PR (#14523) said no fused SDPA kernel takes a mask together with
mismatched head counts and that cuDNN raises on `enable_gqa`. Neither is
right: torch's cuDNN kernel accepts both, and the `enable_gqa` raises in
`attention_dispatch.py` are in the context-parallel ops, not the backends.
What actually happens is that torch tries its kernels in a priority order
that changes across versions and GPUs, and whenever math comes before cuDNN
an `enable_gqa` + mask call lands on math and materializes the full score
matrix.

Rewrite the GQA section of models.md around that (kernel compatibility
table, why to repeat when there is a mask) and fix the Krea 2 processor
comment to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yiyixuxu

Copy link
Copy Markdown
Collaborator Author

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

@yiyixuxu
yiyixuxu merged commit bc096b5 into main Aug 26, 2026
16 checks passed
@yiyixuxu
yiyixuxu deleted the gqa-cudnn-docs branch August 26, 2026 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models size/S PR with diff < 50 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants