Repository navigation
Add ⚡️Sana: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer Support #5785
Description
Feature Idea
🐱 Sana Model Card
Model
We introduce Sana, a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution.
Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.
Source code is available at https://github.com/NVlabs/Sana.
Model Description
- Developed by: NVIDIA, Sana
- Model type: Linear-Diffusion-Transformer-based text-to-image generative model
- Model size: 1648M parameters
- Model resolution: This model is developed to generate 1024px based images with multi-scale heigh and width.
- License: CC BY-NC-SA 4.0 License
- Model Description: This is a model that can be used to generate and modify images based on text prompts.
It is a Linear Diffusion Transformer that uses one fixed, pretrained text encoders (Gemma2-2B-IT)
and one 32x spatial-compressed latent feature encoder (DC-AE). - Resources for more information: Check out our GitHub Repository and the Sana report on arXiv.
Model Sources
For research purposes, we recommend our generative-models Github repository (https://github.com/NVlabs/Sana),
which is more suitable for both training and inference and for which most advanced diffusion sampler like Flow-DPM-Solver is integrated.
MIT Han-Lab provides free Sana inference.
- Repository: ttps://github.com/NVlabs/Sana
- Demo: https://nv-sana.mit.edu/
🧨 Diffusers
Uses
Direct Use
The model is intended for research purposes only. Possible research areas and tasks include
-
Generation of artworks and use in design and other artistic processes.
-
Applications in educational or creative tools.
-
Research on generative models.
-
Safe deployment of models which have the potential to generate harmful content.
-
Probing and understanding the limitations and biases of generative models.
Excluded uses are described below.
Out-of-Scope Use
The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
Limitations and Bias
Limitations
- The model does not achieve perfect photorealism
- The model cannot render complex legible text
- fingers, .etc in general may not be generated properly.
- The autoencoding part of the model is lossy.
Bias
While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
Existing Solutions
No response
Other
No response
Activity
makisekurisu-jp commentedon Nov 26, 2024
Unfortunately, only the 512-pixel and 1024-pixel versions of the model have been released so far. According to the README, it seems there is a 4096-pixel version as well.
jomcey commentedon Nov 26, 2024
Pretty sure that's just referring to generating 4096x4096 images, not that there's a separate model.
goodgutter commentedon Dec 2, 2024
is there a way to run this already in comfyui?
ltdrdata commentedon Dec 2, 2024
There is an unofficial implementation if you are interested.
https://github.com/zmwv823/ComfyUI-Sana
zmwv823 commentedon Dec 10, 2024
Now use official custom-node ComfyUI_ExtraModels instead.
makisekurisu-jp commentedon Dec 10, 2024
Exciting news!
ashrafbay commentedon Dec 11, 2024
Full of errors for windows I could not run it
zmwv823 commentedon Dec 11, 2024
Waiting for complete, it's in progress.
0.6b works.
city96 commentedon Dec 12, 2024
I'm currently doing a full rewrite of ExtraModels to try and get better support on both PixArt and Sana.
Progress is being tracked on this PR: city96/ComfyUI_ExtraModels#92
ashrafbay commentedon Dec 12, 2024
This is so cool. Thanks for your work.
makisekurisu-jp commentedon Dec 24, 2024
As of now, there does seem to be a 4K version of the model.
(🔥 New) [2024/12/20] 1.6B 2K resolution Sana models are released: [BF16 pth] or [BF16 diffusers]. 🚀 Get your 2K resolution images within 4 seconds! Find more samples in Sana page.
julien-blanchon commentedon Jan 8, 2025
Is inpainting available for SANA ?
clementine-of-whitewind commentedon Jan 20, 2025
Please !