iframe-proxy

DavidBert · 2026-06-12T10:22:13Z

What does this PR do?

Adds a pixel-space variant of PRX that denoises raw RGB directly (no VAE), conditioned on a Qwen3-VL text encoder:

PRXTransformer2DModel: new optional config args bottleneck_size (two-layer img_in projection for large patch dims) and resolution_embeds (PRXResolutionEmbedder conditions the timestep modulation on the latent resolution)
PRXPipeline: support for subclass-tuned tokenizer max length, light text cleaning, x-prediction flow matching (x0 -> velocity conversion), and non-unit initial noise scale
PRXPixelPipeline: thin subclass wiring the above together (vae optional/None, vae_scale_factor=1, 1024px default)
conversion script support for the pixel checkpoint format
registration in init files + dummy objects, docs autodoc entry, fast pipeline tests

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline?
Did you read our philosophy doc (important for complex PRs)?
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

@dg845

HuggingFaceDocBuilderDev · 2026-06-13T06:48:00Z

dg845

Thanks for the PR! Left an initial design review :).

github-actions · 2026-06-13T08:34:42Z

Hi @DavidBert, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

dg845 · 2026-06-16T04:05:24Z

+        scheduler: FlowMatchEulerDiscreteScheduler,
+        text_encoder: PreTrainedModel,
+        tokenizer: AutoTokenizer | PreTrainedTokenizerBase,
+        vae: AutoencoderKL | AutoencoderDC | None = None,


I think the vae should be removed rather than retained as an optional component for PRXPixelPipeline. My understanding is that if a non-trivial VAE is supplied with either vae_scale_factor > 1 or self.vae.config.latent_channels != self.transformer.config.in_channels, the code will break.

Good catch — agreed. vae has been removed entirely from PRXPixelPipeline (the __init__ signature, register_modules, _optional_components, and model_cpu_offload_seq). vae_scale_factor is now a simple property returning 1. The Hub checkpoint's model_index.json also needs the "vae": [null, null] entry removed so from_pretrained continues to work — updating that separately.

dg845

Thanks for iterating! Left some follow up comments.

Adds a pixel-space variant of PRX that denoises raw RGB directly (no VAE), conditioned on a Qwen3-VL text encoder: - PRXTransformer2DModel: new optional config args `bottleneck_size` (two-layer img_in projection for large patch dims) and `resolution_embeds` (PRXResolutionEmbedder conditions the timestep modulation on the latent resolution) - PRXPipeline: support for subclass-tuned tokenizer max length, light text cleaning, x-prediction flow matching (x0 -> velocity conversion), and non-unit initial noise scale - PRXPixelPipeline: thin subclass wiring the above together (vae optional/None, vae_scale_factor=1, 1024px default) - conversion script support for the pixel checkpoint format - registration in __init__ files + dummy objects, docs autodoc entry, fast pipeline tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

dg845

Thanks for your work on this PR!

Adds a pixel-space variant of PRX that denoises raw RGB directly (no VAE), conditioned on a Qwen3-VL text encoder: - PRXTransformer2DModel: new optional config args `bottleneck_size` (two-layer img_in projection for large patch dims) and `resolution_embeds` (PRXResolutionEmbedder conditions the timestep modulation on the latent resolution) - PRXPipeline: support for subclass-tuned tokenizer max length, light text cleaning, x-prediction flow matching (x0 -> velocity conversion), and non-unit initial noise scale - PRXPixelPipeline: thin subclass wiring the above together (vae optional/None, vae_scale_factor=1, 1024px default) - conversion script support for the pixel checkpoint format - registration in __init__ files + dummy objects, docs autodoc entry, fast pipeline tests Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

github-actions Bot added size/L PR with diff > 200 LOC documentation Improvements or additions to documentation models tests utils pipelines and removed size/L PR with diff > 200 LOC labels Jun 12, 2026

dg845 self-requested a review June 13, 2026 05:21

dg845 reviewed Jun 13, 2026

View reviewed changes

Comment thread src/diffusers/pipelines/prx/pipeline_prx.py Outdated

dg845 reviewed Jun 13, 2026

View reviewed changes

Comment thread src/diffusers/pipelines/prx/pipeline_prx.py Outdated

dg845 reviewed Jun 13, 2026

View reviewed changes

Comment thread src/diffusers/pipelines/prx/pipeline_prx_pixel.py Outdated

dg845 reviewed Jun 13, 2026

View reviewed changes

Comment thread tests/pipelines/prx/test_pipeline_prx_pixel.py Outdated

dg845 reviewed Jun 13, 2026

View reviewed changes

DavidBert force-pushed the prx-pixel branch from 233b999 to 71fcc14 Compare June 15, 2026 13:41

github-actions Bot added the size/L PR with diff > 200 LOC label Jun 15, 2026