{{ message }}
This repository was archived by the owner on Jan 24, 2024. It is now read-only.
Repository navigation
This repository was archived by the owner on Jan 24, 2024. It is now read-only.
crash when running mosaicml/mpt-7b-* models: KeyError: 'attention_mask' #213
Description
from basaran.model import load_model
model = load_model('mosaicml/mpt-7b-storywriter', trust_remote_code=True, load_in_8bit=True,)
for choice in model("once upon a time"):
print(choice)Traceback (most recent call last):
File "/home/taras/Documents/ctranslate2/basaran/run.py", line 7, in <module>
for choice in model("once upon a time"):
File "/home/taras/Documents/ctranslate2/basaran/.venv/lib/python3.9/site-packages/basaran/model.py", line 73, in __call__
for (
File "/home/taras/Documents/ctranslate2/basaran/.venv/lib/python3.9/site-packages/basaran/model.py", line 233, in generate
inputs = self.model.prepare_inputs_for_generation(
File "/home/taras/.cache/huggingface/modules/transformers_modules/mosaicml/mpt-7b-storywriter/8667424ea9d973d3c01596fcbb86a3a8bc164299/modeling_mpt.py", line 280, in prepare_inputs_for_generation
attention_mask = kwargs['attention_mask'].bool()
KeyError: 'attention_mask'
Activity
[-]crash when running storywriter model[/-][+]crash when running storywriter model: KeyError: 'attention_mask'[/+][-]crash when running storywriter model: KeyError: 'attention_mask'[/-][+]crash when running mosaicml/mpt-7b-* models: KeyError: 'attention_mask'[/+]tarasglek commentedon Jun 12, 2023
Same thing happens with
mosaicml/mpt-7b-instructfardeon commentedon Jun 13, 2023
It appears that the error originates from the internal code of MPT. We will conduct further testing.