Thanks for fixing this. I took the chance to delete a redundant pretraining config and fix some other parameters. Also, there are 5 toxicity electra configs. You have changed one learning rate to 1e-3, the others are 1e-5. Is there a reason for (a) having 5 different toxicity electra configs (but none for other OPT finetuning tasks) and (b) for having different learning rates?

They are used for different fine-tuning tasks (classification vs regression) and using different parameters, however, have deleted some of them. The remaining should stay.