{{ message }}
Repository navigation
Unable to load larger models #3623
Description
Describe the bug
Running
StableDiffusionPipeline.from_ckpt("path_to_safetensorfile")
or the
./scripts/convert_original_stable_diffusion_to_diffusers.py
Causes some weights to be removed from the models. The model does not work as well as it should. The results from same settings (seeds) is different than results on Automatic1111 and any other tools
Reproduction
Download any model larger than 2gb from civitai.com
https://civitai.com/models/6755/cetus-mix < this model works
Run
StableDiffusionPipeline.from_ckpt("path_to_safetensorfile")
Logs
global_step key not found in model
In this conversion only the non-EMA weights are extracted. If you want to instead extract the EMA weights (usually better for inference), please make sure to add the `--extract_ema` flag.
Some weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel: ['vision_model.encoder.layers.5.self_attn.out_proj.bias', 'vision_model.encoder...
...]
- This IS expected if you are initializing CLIPTextModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS NOT expected if you are initializing CLIPTextModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). `text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
/home/rip/.local/lib/python3.11/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
### System Info
- `diffusers` version: 0.17.0.dev0
- Platform: Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.35
- Python version: 3.11.3
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Huggingface_hub version: 0.14.0
- Transformers version: 4.28.1
- Accelerate version: 0.18.0
- xFormers version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
Activity
JongSeok-Kim commentedon Jun 8, 2023
I have same problem
patrickvonplaten commentedon Jun 11, 2023
Could you add a link of a model that cannot be correctly converted here?
liuchangzong commentedon Jun 21, 2023
Why this issue is closed?
I'm confused of how to convert .safetensors file to diffusers model by convert_original_stable_diffusion_to_diffusers.py.
I can't convert the model that I have downloaded from civitai .
Have you solved this problem?
VincentNeemie commentedon Jun 21, 2023
Same here, i was using it fine yesterday however today it doesn't work anymore.
Apparently it has to do with the Omegaconf raising an error exception when the global key is not found.
Funny enough, i have a saved version of diffusers on my repositories of a few weeks ago and that one works without a problem, even though it still says the global keys are not found, it just ignores the issue...
I will post a solution to it #3843
patrickvonplaten commentedon Jun 28, 2023
I can't reproduce the error because as you say https://civitai.com/models/6755/cetus-mix works. It would be very nice if you could post a model that does not work.
@VincentNeemie, adapter your PR a bit, but there seems to have indeed been a bug
VincentNeemie commentedon Jun 28, 2023
Thanks for the adaptation, it elegantly handled this issue
I rushed the solution a bit and I must confess that it could have been done better, as your adaptation shows! But things were not working and I wanted to provide an insight to this issue as fast as possible
The reason why the bug can't be reproduced anymore is because you fixed it last week #3848 albeit a day later I posted my solution #3843
To reproduce this error, you would have to go to the merge that was created from this PR #3815 , which was created at 17 June but committed at 21 June, the day the errors appeared, any model from civitai would suffice. (-But for reference: https://civitai.com/models/8124/ , https://civitai.com/models/43331/ , https://civitai.com/models/4384/ )
Thank you for taking the time to merge my PR, I will be looking forward to contribute more to this community