Linked issue or supporting maintainer

Fixes #178052

BC-breaking?

No. With torch.use_deterministic_algorithms(True), CUDA ctc_loss backward used to raise and now succeeds. Gradients on large inputs may differ from the non-deterministic path in the last bits (due to reduction order).

Summary

Currently, only the large-input CUDA CTC backward used atomicAdd (the nonblank kernel); the small-input kernel was already deterministic, but ctc_loss_backward_gpu raised for every size.
This PR removes the blanket alert and, in deterministic mode, replaces the nonblank atomic update with label grouping (stable sort + binary-searched offsets) and a fixed-order warp reduction.

Benchmarks

Backward latency in milliseconds on an NVIDIA RTX 3080 (10 GB), CUDA 12.8, cuDNN 9.8.0 (cuDNN disabled for these runs, so the native kernels are measured). Each number is the median of 3 rounds of 10 calls after 5 warmup calls, timed with CUDA events [state whether each round reports the mean of its 10 calls]. Shapes are (T, B, C, L): input length, batch size, vocabulary size, and target length.

* Uses the small-input kernel in both modes, which is unchanged, so the difference reflects measurement noise.

On large inputs the deterministic path takes 0.92x-1.32x the time of the non-deterministic path.

For reference, the native deterministic path versus cuDNN (float32 only, forward + backward combined, since cuDNN computes gradients in the forward pass):

cuDNN is only used when blank == 0, targets are contiguous int32 on CPU, log_probs is float32, all input lengths equal T, and target lengths are below 256. The native path remains the fallback for all other inputs.

Agent details (if applicable)

Worked with GPT-6.1-Sol and Claude Code Sonnet 5.5