New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fsdp-qlora yi-34B-chat throw error " ValueError: Cannot flatten integer dtype tensors" #3470
Comments
have seen exist issue written in March,but i cannot get any useful info to find out why this error came,hoping your suggestion |
please provide your version of accelerate and bitsandbytes |
@hiyouga accelerate==0.28.0 bitsandbytes==0.43.0 ,Do these versions have any problems?hoping your suggestion |
did you use the latest code? |
accelerate 0.29.3 |
@hiyouga sorry,my answer is so late this case, using newest llama_factory code, it work currently right now |
Reminder
Reproduction
CUDA_VISIBLE_DEVICES=0,1 accelerate launch
--config_file config.yaml
src/train_bash.py
--stage sft
--do_train
--model_name_or_path /workspace/models/Yi-34B-Chat
--dataset law_with_basis
--dataset_dir data
--template default
--finetuning_type lora
--lora_target q_proj,v_proj
--output_dir /workspace/ckpt/Yi-34B-Chat-sft
--overwrite_cache
--overwrite_output_dir
--cutoff_len 1024
--per_device_train_batch_size 1
--per_device_eval_batch_size 1
--gradient_accumulation_steps 8
--lr_scheduler_type cosine
--logging_steps 10
--save_steps 100
--eval_steps 100
--evaluation_strategy steps
--load_best_model_at_end
--learning_rate 5e-5
--num_train_epochs 3.0
--max_samples 3000
--val_size 0.1
--quantization_bit 4
--plot_loss
--fp16
config.yaml
Expected behavior
fsdp qlora yi-34B-chat
System Info
transformers 4.39.3 torch 2.1.2 cuda 121 python3.8
Others
The text was updated successfully, but these errors were encountered: