evaluation

[ZHANGJINKUI]

System Info

python eval.py --model hf --model_args pretrained=/mnt/sdb/zjk/llama2/llama-recipes/Llama-2-7b-hf,dtype="float",peft=/mnt/sdb/zjk/llama2/llama2-lora --tasks hellaswag --num_fewshot 10 --device cuda:0 --batch_size 8
error：
2024-04-02:06:58:12,156 ERROR [eval.py:226] An error occurred during evaluation: module 'lm_eval.tasks' has no attribute 'initialize_tasks'

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

evaluation was changed？
how can i evaluate finetuned model with lora now？

Error logs

2024-04-02:06:58:12,156 ERROR [eval.py:226] An error occurred during evaluation: module 'lm_eval.tasks' has no attribute 'initialize_tasks'

Expected behavior

evaluate finetuned model with lora and the accuracy of specific dataset with cola

[LHQUer]

Hello! Have you solved this issue? I have the this question as same as yours. If you have solved this question, Please tell me how to do that. Thanks bro.

[mreso]

Hi @LHQUer @ZHANGJINKUI

what version of lm_eval are you using?