Pull requests

• [AutoEP] Ensure consistent order of partition groups in all-gather when ZeRO-3

  #8876·jinyouzhi opened Oct 21, 2026Contributor·

  ·
  1

• Fix ZeRO-3 external parameter discovery for resident weights

  #8875·DimensionSTP opened Oct 21, 2026·

  ·
  1

  4

• Fix double-transpose crash loading fused QKV checkpoints (megatron_v2)

  #8873·udsy19 opened Oct 21, 2026·

  ·
  1

  4

• [checkpoint] universal state keys for non-Adam optimizers (fix #8608)

  #8870·Asthenia0412 opened Oct 20, 2026·

  ·
  1

• Report skipped optimizer steps from the non-ZeRO optimizer wrappers

  #8869·0z5a opened Oct 20, 2026Contributor·

  ·
  6

• Make the Triton device guards accelerator-agnostic

  #8868·VenusTZZ opened Oct 20, 2026Contributor·6 tasks done·

  ·
  6

• Move test_klxpu_accelerator.py to tests/unit/v1/accelerator/

  #8867·GuCarpenter opened Oct 20, 2026Contributor·

  ·
  1

• Keep the tensors a model names for fp32 in fp32 in init_inference

  #8864·udsy19 opened Oct 20, 2026·

  ·
  1

  4

• Fix subgroup communication bandwidth calculation

  #8862·marcusacosta opened Oct 19, 2026·

  ·
  1

  3

• AutoTP: sequence sharding for training

  #8860·pengdurice opened Oct 19, 2026Contributor·

  ·

• AutoTP: shard an untied input embedding along the vocabulary (`vocab_…

  #8859·pengdurice opened Oct 19, 2026Contributor·

  ·
  1

• Fix BF16 full-gradient updates to use the accumulation buffer

  #8856·mottopanikeiku opened Oct 18, 2026·

  ·
  4

• Scale ZeRO-3 __avg_scatter_grads by sequence_parallel_size

  #8854·deep1blue opened Oct 18, 2026·

  ·
  1

  1

• Measure first-call and post-update costs of continuous rollout CUDA Graphs

  #8853·nathon-lee opened Oct 18, 2026Contributor·

  ·

• Support Transformers 5.20 cache query offsets in DeepSpeedStaticCache

  #8852·nathon-lee opened Oct 18, 2026Contributor·

  ·

• [AutoTP] Support frozen AutoTP models and manage AutoTP mode per model

  #8851·jinyouzhi opened Oct 17, 2026Contributor·

  ·

• AutoEP + AutoTP folding: Universal Checkpointing under ZeRO-3

  #8848·pengdurice opened Oct 16, 2026Contributor·

  ·

• AutoEP: support combine_impl="fused_weighted_sum" with AutoEP+AutoTP folding

  #8847·pengdurice opened Oct 16, 2026Contributor·

  ·

• AutoTP: send CPU copies in the first-batch consistency check

  #8845·pengdurice opened Oct 16, 2026Contributor·

  ·

• Universal Checkpointing: read AutoEP ZeRO-3 optimizer files one at a time

  #8844·pengdurice opened Oct 16, 2026Contributor·

  ·

• Keep the parameters a model names for fp32 in fp32 under ZeRO-3

  #8843·pengdurice opened Oct 16, 2026Contributor·

  ·

• ZeRO-3: reuse the weight partition as the master copy when it already has the master dtype

  #8842·pengdurice opened Oct 16, 2026Contributor·

  ·

• Peng add cpu adam fused step v1

  #8841·pengdurice opened Oct 16, 2026Contributor·

  ·
  1

• ZeRO-3: copy offloaded master weights back to the device through pinned buffer

  #8840·pengdurice opened Oct 16, 2026Contributor·

  ·
  1

• [OPSD] Request last-position logits only in HybridEngine rollout prefill

  #8839·jinyouzhi opened Oct 16, 2026Contributor·

  ·
  3