You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
FLUX LoRA Fine-Tuning: Border Outpainting and Pixel-Level Misalignment Issues in Image Editing #1685
Why does the flux.2-klein-base-9b model, when fine-tuned via LoRA on pixel-aligned pre- and post-edited image pairs with prompts for an image editing task, exhibit slight border expansion (outpainting), a failure to preserve pixel-level structural consistency, and minor detail distortions during inference?
This pattern often comes from train/infer mismatch on how the source–target pair is resized and conditioned, not from LoRA rank alone.
Worth checking in order:
Pair geometry — are pre/post images exactly the same H×W before encode? Letterbox vs center-crop vs stretch will teach the model a soft “border expand”.
Encode path — if source and target go through VAE with different pad/crop, latent grids won’t share the same pixel lattice and you get the slight structural drift you describe.
Noise / CFG — edit LoRAs are sensitive to train noise scale and to high CFG at inference; try identity or near-identity prompts with lower CFG and see if the border still grows.
Dataset edge padding — constant-color pads around the canvas are a classic outpainting prior.
A quick A/B: same-size center crop (no pad), train a short run, then infer with identical resolution + low CFG. If the halo shrinks, it’s preprocessing; if not, share your train YAML (resolution, bucket, noise, CFG) and one sample pair sizes and we can narrow further.
Thanks for the suggestions! The border expansion issue has been resolved after adjusting the preprocessing (ensuring consistent geometry). However, pixel-level alignment remains a challenge.
I have verified that the training and inference settings are identical, but I’ve observed a significant discrepancy between frameworks using the official flux.2-klein-base-9b model:
With the same prompt and identical parameters, DiffSynth results show much less deviation from the source image, whereas Diffusers produces significantly larger changes.
Activity
tutao0123 commentedon Sep 14, 2026
This pattern often comes from train/infer mismatch on how the source–target pair is resized and conditioned, not from LoRA rank alone.
Worth checking in order:
A quick A/B: same-size center crop (no pad), train a short run, then infer with identical resolution + low CFG. If the halo shrinks, it’s preprocessing; if not, share your train YAML (resolution, bucket, noise, CFG) and one sample pair sizes and we can narrow further.
lllllialois commentedon Oct 14, 2026
Thanks for the suggestions! The border expansion issue has been resolved after adjusting the preprocessing (ensuring consistent geometry). However, pixel-level alignment remains a challenge.
I have verified that the training and inference settings are identical, but I’ve observed a significant discrepancy between frameworks using the official flux.2-klein-base-9b model:
With the same prompt and identical parameters, DiffSynth results show much less deviation from the source image, whereas Diffusers produces significantly larger changes.