Hello, I'm a TTS researcher who heavily relies on transformers library for my research work.
Recently, I found a simple but somewhat critical bug for model researcher while I using Higgs Audio V2 model.

So, I'd like to report and PR this fix (without Issue).
Below is the content what I am reporting:

HiggsAudioV2ForConditionalGeneration.forward allocated the expanded audio label tensor with a hardcoded last dimension of 8:

audio_labels_expanded = input_ids.new_ones((*input_ids.shape[:2], 8)) * -100

while every surrounding computation is driven by
self.config.num_codebooks: audio_logits is reshaped to (..., num_codebooks, codebook_size) and the per-codebook loss loop iterates over range(self.config.num_codebooks).

8 is just the default of HiggsAudioV2Config.num_codebooks, so the two only agree by coincidence. With any other value the next line fails to broadcast:

RuntimeError: shape mismatch: value tensor of shape [20, 2] cannot be broadcast to indexing result of shape [20, 8]

This happens for any num_codebooks != 8, smaller or larger, the loss loop is never reached.

Replace the literal with self.config.num_codebooks.
Applied to modular_higgs_audio_v2.py and regenerated into modeling_higgs_audio_v2.py.

audio_labels is not passed anywhere in tests/, so this branch was dead code in CI. I added test_forward_with_audio_labels; the tester already uses num_codebooks=2, so it fails on the current code and passes with the fix.

What does this PR do?

Code Agent Policy

• (First-time contributors only): I confirm that this PR description and code is not written by an LLM or code agent

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline and the
  Pull Request checks?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes according to the guidelines?
• Did you write any new necessary tests?

Who can review?

Models:

• audio models: @eustlb @ebezzam